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PART I 
INTRODUCTION 



CHAPTER 1 
PURPOSE OF THE REVIEW 



A. Defining Institutional Change for Limited English Proficient Students 

The Improving America's Schools Act, the 1994 reauthorization of the Elementary and 
Secondary Education Act including Title VII programs, defines important new emphases in 
federal support for effective instructional services for children and youth. The new 
legislation supports the goal of high-quality education for all students by building the 
capacity of schools to help students meet challenging content and performance standards. 
A central component in this effort is a systemic approach to the improvement and reform 
of educational programs that serve limited English proficient students. In fact, two new 
program categories, Comprehensive School Grants and Systemwide Improvement Grants, 
specifically address the need for comprehensive and integrated approaches to the education 
of those students. The new program categories reflect a broader view of effective 
instruction, in which the whole school working together with the local community is 
recognized as the key source of input for both students and for educators. This whole 
school perspective and systemic approach is consistent with recent research and with the 
eight National Education Goals. It is also consistent with the emphasis on change processes 
and systemic change characteristic of the more recent education reform efforts. 

This literature review marks an initial step toward the design of research on the 
implementation of effective change efforts related to limited English proficient students. The 
research, which will be carried out through the Department of Education, Office of Bilingual 
Education and Minority Languages Affairs, will track change efforts related to services for 
limited English proficient students over the next several years. The purpose of this research 
is to learn how best to promote the design and implementation of effective school contexts 
and services for those students. Essentially, the goal is to identify how change efforts can 
be designed and implemented to incorporate language minority and limited English 
proficient students as equal participants with other students within educational reform. 

The information to be obtained through the study will answer questions such as: How can 
the federal government — through Title VII programs, for example — be most effective in 
promoting educational change that will benefit limited English proficient students? What 
should be the content of federal efforts? What should be the process for promoting change 
efforts? 

The Special Issues Analysis Center was requested to carry out this literature review on 
institutional change to facilitate and contribute to the development of a framework for 
possible research designs. Broadly, the questions addressed by the literature review are the 
following: 

■ What specific components should be included within a framework that 
outlines the content of change within institutions that serve limited English 
proficient students? 
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■ What guiding theories or principles on the institutional change process are 
identified in the literature? 

■ What context or implementation factors can be identified and examined as key 
elements relevant to the change process within institutions serving limited 
English proficient students? 

■ What interactions among variables defining the content and process of change 
are indicated by the literature? What does the literature indicate are the 
implications of these interactions for institutions involved in implementation 
of change? 

■ What recommendations for methodologies applicable to measuring change can 
be identified in the research literature? 

B. Research on Educational Change and Limited English Proficient Students 

The study of change, although now taking a more comprehensive and systemic direction, 
is still relatively new. Progress is evident, however, in that educators have come to 
understand more about the nature of change. They have learned, over the past two decades, 
that implementation is key and that how a change is put into practice is central to how well 
the change will succeed (Fullan, 1991). The setting in which the change is implemented also 
affects how the change will be received and implemented, since local conditions and 
contexts play a role in determining the final shape of the change (Berman et al, 1975). 

To a large extent, however, issues related to language minority students in particular have 
not been specifically included within research on school-wide reform and change (Gandara, 
1994; Valadez, 1989). For example, Gandara's review of articles (1994) found that in the 
previous ten years, only about 4 percent of articles on school reform published in the Phi 
Delta Kappan mentioned students from non-English backgrounds. In conducting our review 
of the literature, we searched for references in the ERIC database and found 4,148 articles 
(1990 to present) that were described as concerning educational change in grades K-12; of 
those references, only 138 also included bilingual or English as a Second Language (ESL) or 
language minority as a descriptor. It is clear from these examples that the majority of 
studies do not incorporate a specific concern with language minority students or schools 
serving such students. These statistics reflect perhaps an underlying assumption that the 
findings are relevant to all participants in reform. While it is true that findings from any 
study of school change are likely to apply to schools that serve limited English proficient 
students, there are also likely to be issues that are specific to those schools. For example, 
there is a considerable body of research on the development of school professional 
communities, in which teachers collaborate and share information and concerns regarding 
students and instruction. Generally absent, however, are discussions of the issues involved 
when teachers from different programs or perspectives begin to work together, as would 
occur when mainstream teachers collaborate with bilingual classroom or ESL teachers. 
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We need to explore these and other relevant issues to further inform school change efforts 
within schools that serve limited English proficient students. Recent data on students show 
that the number of students from language minority backgrounds is increasing and that the 
number of schools enrolling at least one limited English proficient student is also growing 
(e.g., Fleischman and Hopstock, 1993). States participating in the SEA Title VII grant 
program report a total of approximately 2.7 million students identified as limited English 
proficient for the 1992-93 academic year (Henderson, Donly, and Strang, 1994) and the 
California State Department of Education (1992) reported that the number of limited English 
proficient students in 1991 was almost double the 1985 figure. Since these trends in growth 
are expected to continue, issues related to the instruction of limited English proficient 
students are expected to become increasingly important to schools. There is a need, 
therefore, to examine school reform and change efforts with a focus on such issues. 



C. An Approach to the Description of Institutional Change Efforts 

We have stated that the objective of this literature review is to provide input to the design 
of a study of institutional change efforts that affect limited English proficient students by 
identifying key findings and describing key components of change efforts. To describe and 
track change, we first need to be able to describe what it is that is changing. 

At its most basic level, change has always been the objective of education in that education 
is directed toward change within students — toward adding to and revising a student's 
understanding and knowledge. At that level, there has been an implicit understanding that 
change involves a particular area of content or skills, that it involves a process, and that it 
will take time. Much of the educational research that has been conducted has focused on 
identifying classroom instructional approaches and practices that will be effective in 
promoting change. Within the area of instruction for language minority students, much has 
been learned about the process of learning and how to best promote learning. For language 
minority students, the use of the native language for instruction or to support instruction, 
the use of English within meaningful contexts as in content-based ESL approaches (e.g., 
Short, 1993), the incorporation of students' culture and background within instruction (e.g., 
Moll, 1990), and the provision of challenging content to students rather than rote-learning 
based curriculum (e.g., Warren and Rosebery, 1990) are examples of findings that have 
helped to define effective instruction for limited English proficient students. 

Recent research has begun to focus on the nature of instruction as one part of a complex of 
factors that define effective instructional environments for students. In particular, this focus 
has been on factors within the school which ultimately affect the teacher-student relationship 
and instructional outcomes and on change at several interacting levels within the 
educational system. Researchers have described educational change as a multi-level, multi- 
dimensional process (Fullan, 1991; Rowan, 1991; Raudenbush & Bryk, 1988-89) through 
which the nature of the school community, linkages among school staff, linkages with the 
home and community, student background, culture, and experience all play a role in 
determining student achievement, the ultimate goal of change efforts. 



From this perspective, the description of change efforts is a particularly complex task. Any 
research on change thus must require careful definition of the nature of the change efforts 
to be studied and of the types of variables to be included within the scope of the study. The 
findings of this literature review are expected to help guide this definition process and to 
identify key variables for a study of school change affecting limited English proficient 
students. 

Our starting premise, based on a basic understanding that has been gained through the 
research on change, is that the description of any change effort involves description of both 
the content of the change and the process of change. The fact that change is both content 
and process has not always been recognized, with the result that there have been essentially 
two perspectives on change at different times. The first perspective regards change as 
content, describes chmges in terms of the characteristics or attributes of innovations, 
settings, and participants, and looks for relationships among these attributes to explain the 
success or failure of a change effort. The second perspective regards change as process, 
looks at change as a flow or series of processes that are described in terms of actions or 
behaviors that often involve interactions among individuals or groups in different roles. 

What gets confusing is that within the two perspectives similar words are often used to 
describe different things or different terms are used to describe the same things. The 
primary source of confusion, however, is that both perspectives are often observed but 
without making the distinction between the content of the change and the process of the 
change. 

For a study of change, however, it is critical to have a clear approach to examining and 
exploring relationships among components of both the content and process of change. 
Without doing so we will lose the richness of each perspective and we will be less likely to 
produce findings that can be used to guide further practice and inform policy. Therefore, 
in our review of the literature, we have chosen to discuss separately the content of change 
efforts and the process of change efforts. 



D. Plan of the Literature Review 

In our review of the literature, we are attempting to clarify the key components and findings 
related to the content and process and to examine ways in which these interact. The specific 
research questions identified for this review are as follows: 

(1) What is the content or nature of the change being implemented? How should 
this content be defined? 

(2) How is the change being implemented? That is, what is the process of change 
and how can this process be measured? 

Finally, all change efforts are expected to result in benefits in terms of student outcomes. 
The specific types of changes implemented within certain contexts and involving identified 
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participants and their interactions will need to be understood in order to define how student 
outcomes result from identified change efforts. The discussion of the content of change and 
the process of change should thus inform further research and practice on school change 
and, in particular, research on school change affecting limited English proficient students. 

Part I of the review (Chapters 2 and 3) provides background for the focused review on 
school change. Chapter 2 presents an overview of conceptual models of change and 
perspectives on change. Chapter 3 presents a synopsis of practices and findings within 
districts that are undergoing significant district-wide change efforts. 

Parts II and III of the review together present a discussion of the nature of change efforts. 
Part II (Chapters 4, 5, and 6) presents a framework for describing the content of change in 
terms of the goals of the change and the areas of focus of the change within the school. Part 
III (Chapters 7, 8, and 9) presents a framework for describing the process of change. The 
process is defined in terms of the stages of the change process and the characteristics of the 
change process. 

Part IV (Chapter 10) presents the summary framework for the definition of school change 
efforts affecting limited English proficient studenL. The chapter also presents a discussion 
of methodological and measurement issues within a study of change and makes 
recommendations regarding the design of a study on institutional change. 
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CHAPTER 2 



UNDERSTANDING THE NATURE OF CHANGE 



A. Introduction 

The early decades of federal involvement in educational change were a period of optimism. 
The late 1950s' Sputnik-induced science and math programs, followed by passage of the 
Elementary and Secondary Education Act (ESEA) and its first few reauthorizations and 
expansions through the early 1970s, led to local breaks with the status quo, improved 
practice in the classroom, and improved student performance— or so it was believed. It 
seemed so easy. As a nation, we could develop better educational products, methods, 
approaches, or technologies. Educators seeking better ways to educate our children would 
adopt the educational tools that they needed. 

We noted a few operational problems with this model earl/ on, particularly that educators 
had trouble obtaining information about new approaches and that developers were 
sometimes not in touch with the needs of the educators, so their products went unused. We 
dealt with those problems, creating ERIC as a comprehensive information resource and 
revising our views of the world to include the importance of two-way linkages between the 
developer community and the user community so that needs could flow as easily as 
products. In response to other difficulties, we kept tweaking the system; we added 
"facilitators" to help educators select products, and we added technical assistance providers 
to help with the relatively esoteric sides of operations, such as evaluation. And we added 
new programs and modified existing ones to expand the range of available choices. But it 
still did not seem to work. 

What went wrong? By the mid-1970s, researchers were seeking answers in several 
directions and often came to the same general conclusions: (1) our assumptions, and 
therefore our models and policies, were wrong; (2) change is sometimes fast, sometimes 
slow, but generally change is occurring; and (3) the effects that changes produce have less 
to do with the products that are available than with the process of making the changes. 
Subsequent research has modified these points, finding that some of our earlier assumptions 
were not completely wrong after all, although the circumstances in which they hold true are 
limited. In the pages that follow, we review the various conceptual models and perspectives 
that have guided the research and policy related to educational change. In doing so, we 
hope to come to a better understanding of the nature of change and effective methods of 
influencing change in schools. 

B. Approaches to Describing Educational Change 

Models of educational change are dominated by efforts to develop explanations or 
approaches that are useful in planning and managing change. During the 1960s, several 
researchers (Katz and Kahn, 1966; Havelock, 1971; Chin and Benne, 1969) developed 
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taxonomies for describing educational change. For example, Chin's taxonomy identifies 
three approaches: 

■ Empirical-rational; 

n Power-coercive; and 

■ Normative-reeducative. 



1. Empirical-rational Approach 

According to Chin and Benne (1969), an empirical-rational approach involves the 
empirical demonstration of the effectiveness of new knowledge, which therefore will 
be rationally adopted. Included in this approach are the traditional research, 
development, diffusion, and adoption model and the agricultural extension agent 
model. Possible difficulties that could result from the selection of this approach for 
implementing change in educational settings include: "(a) ignoring the new idea, (b) 
resisting or rejecting the new idea, or (c) modifying the idea or practice in such ways 
that, when put into practice, it has been significantly changed" (Owens, 1987, p.237). 

2. Power-coercive Approach 

The power-coercive approach involves the use of sanctions or at least the threat to 
use them in order to gain compliance from the targeted adopter organizations. 
Restructuring of power is a model of change reflective of this approach. "Satisfying" 
or "adopting workable responses to demands (sufficient to avoid sanctions) but not 
seeking maximally effective responses" (Owens, 1987, p 237) is one likely outcome 
resulting from policymakers' selection of this approach for implementing change in 
educational settings. 

Implicit in both the empirical-rational and power-coercive approac hes are that good 
ideas tend to be created outside the organization, and that organizations not only 
emphasize stability over change but also may actively resist change. Thus, from the 
perspective of both of these approaches, districts and schools must be made to 
change by state or national authorities if any change is to occur. And, to be sure, 
educational organizations have remained markedly resistant to change with these 
approaches. The National Institute of Education's (NIE) Group on School Capacity 
for Problem Solving (1975) found, for example, that: 

"No matter how good the channels which transmit knowledge and 
products to practitioners, it appears that such products will spread 
slowly and see little effective use until schools and districts develop the 
capacity to engage in an active search for solutions to their own 
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problems, to adapt solutions to the particulars of their own situation, 
and equally important, to adapt themselves as organizations to the 
requirements of the selected solutions (p. 5). M 

3. Normative-reeducative Approach 

The NIE findings provide support for Chin's normative-reeducative approach. This 
approach takes into consideration the culture of the organization and posits that the 
culture can be moved to become more productive through collaborative action of the 
individuals within the organization. Concepts of change that are reflective of this 
approach include organizational development and organizational self-renewal. 

C. Metaphors for Describing Educational Change 

Another lens for examining changes in educational organizations is metaphorical. Through 
the 1970s, most models of educational change emphasized one or another of two metaphors, 
with the main difference being whether the change was driven by needs recognized by the 
potential users or by some agency external to the user. 

1. Rational Man Metaphor 

The first could be characterized as the "rational man" metaphor, which is similar to 
the empirical-rational model discussed above and assumes that reasonable people 
would seek "best" solutions to the problems they identified. The "rational man" 
metaphor can probably trace its roots back at least as far as John Locke; this user- 
driven approach to making changes to address specific needs was variously labeled 
as a problem-solving model, a concerns-based adoption model (CBAM), or a research 
and development model. 

2. Technological Diffusion Metaphor 

A second metaphor, which could be labeled "technology diffusion," stemmed from 
a specific historical example, the agricultural extension agent spreading knowledge 
of new and better scientific agricultural practice on a farmer-by-farmer basis. This 
metaphor assumes the farmer (or educator) may not know that there is a problem 
that could be corrected through adoption of a new technique, so the change agent is 
needed to pass this knowledge on. 

Neither the rational man nor diffusion metaphors provided satisfactory explanations for a 
broad range of successful or unsuccessful educational changes, but research involving each 
has left a valuable legacy of attributes of users, resources, and linkage mechanisms that are 
important for understanding aspects of educational change (as discussed elsewhere in this 
review). The reason for their lack of explanatory success is that they did not take into 
account the alternative perspective of educational change as a series or flow of processes that 
can begin at many places and end in many ways. . 
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People began questioning the rational man and diffusion metaphors when the results of the 
first national evaluations of federal educational programs began coming in with inconclusive 
results. Some researchers questioned the quality of the evaluations, some blamed the 
programs, and others found fault with school personnel, particularly when the evaluations 
also reported that some of the programs were hard to identify in the field and those that 
were visible seemed to wither away with the departure of fedeiM dollars. This pattern of 
widespread blame led others to question the models themselves, especially in their failure 
to explain why practice did not change, even though good products were being produced 
by developers and were being adopted by school personnel. The result was a focus on the 
process of implementation and the attributes of programs and systems that made for 
different outcomes at the various stages of the process. 

Implementation modelers did not usually deny the importance of rational man or 
technology diffusion models (or coercion, for that matter), agreeing they helped explain the 
process of adopting an innovation. They pointed out, however, that adoption does not 
automatically lead to implementation or, even further down the road, to institutionalization. 
The implementation models added to the study of educational change a host of political, 
organizational, and other practical matters that are at the heart of school district and school 
operations. The underlying premise was that opening up the organizational "black box" was 
the key to understanding when and why changes occur within districts and schools. 



D. The Rand Change Agent Study 

One of the more significant influences on the development of educational change research 
was the 1973-78 Rand Corporation study of federal programs supporting educational change 
(the Rand Change Agent Study, Berman and McLaughlin, 1975.) 

The two main purposes of the study were (1) to describe the process of implementing 
innovative practices in schools and (2) to identify factors that affect the outcomes of 
innovative practices. The study methodology included a review of the literature on 
educational innovations; a nationwide survey of a sample of 293 local projects funded by 
four federally funded programs (Title II and Title VII of the Elementary and Secondary 
Education Act; the 1968 Vocational Education Act; the Right to Read program); and personal 
interviews with local project staff, and state and federal government officials (McLaughlin, 
1990). 

Findings from the Rand study had a profound effect on how policymakers viewed the 
process of bringing about change. The study report stressed the importance of 
implementation within the process of change and noted the influence of local variability on 
change outcomes. Recently, one of the principal authors and invest gators in the original 
study, Milbrey McLaughlin, reviewed the most significant conclusions in light of recent 
practices and understandings of the change process. In this reassessment, she suggests that 
while some of the original findings remain true today, others require modification. 
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1. 



Study Findings That Remain True 



McLaughlin (1990) listed three main conclusions from the Rand Study that could still 
be considered valid: 

■ Policy cannot mandate what matters. 

■ Implementation strategies dominate outcome. 

■ Local variability is the rule; uniformity is the exception. 

Policy cannot mandate what matters. Although federal policies (and federal funds) 
encouraged local projects to adopt innovative practices in education, only a few of 
the projects studied actually implemented these innovations. And even fewer were 
able to institutionalize the innovations that had been adopted. To implement and 
sustain educational change requires both local capacity, e.g., expertise and resources, 
and local will, i.e., motivation and commitment. Neither capacity nor will is strongly 
affected by policy mandates. 

Implementation strategies dominate outcome. Rand found that policy outcomes are 
significantly influenced by differences in local strategies related to how, when, or 
whether to implement an innovation. The researchers concluded that effective 
implementation strategies were those that were consistent with local needs and 
resources, i.e., strategies that promoted mutual adaptation rather than uniform 
implementation, and that provided teachers with the necessary support to participate 
in the change efforts. The findings indicated that, in particular, district leadership 
and support were necessary for successful implementation and continuation. 
According to the Rand report, ineffective strategies were those that did not encourage 
teacher participation, such as failing to provide ongoing training and support and 
relying on external consultants. 

Local variability is the rule; uniformity is the exception. The local projects that 
were surveyed varied substantially in terms of their educational practices, resources, 
and so on. Such diversity does not affect local capacity to influence change, but it 
does play a role in how change comes about and what the change looks like. 

2. Study Findings That Require Modification 

In her reanalysis, McLaughlin (1990) also reported a number of conclusions or 
interpretations of study findings that should be modified to reflect current knowledge 
of the educational change process. For example: 

■ Policy goals can only be achieved if local initiators actively support the goals 
that have been set. 

■ External agents cannot promote change effectively. 
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■ Policy goals are relevant to classroom teachers. 



Policy goals can only be achieved if local initiators actively support the goals that 
have been set. The importance of initial motivation was overemphasized in the 
original study. Findings that related to effective strategies were interpreted to mean 
that unless local initiators were supportive of the change from the beginning, and 
were determined to carry it out, project goals could not be achieved. The study 
overlooked that individuals who are required to change can become actual advocates. 
In a 1985 study of policy changes associated with the transition of federally funded 
compensatory education programs from Title I of ESEA to Chapter 1 of ECIA 
(Education Consolidation and Improvement Act), McLaughlin, Shields, and Rezabek 
(1985) documented "the importance of local will and the fact that individuals or 
institutions originally opposed to an idea can change their minds" (p. 13). 

External agents cannot promote change effectively. The Rand Change Agent Study 
had concluded that external agents, i.e., external consultants or externally specified 
projects, were too far removed from— and not sufficiently responsive to— local needs 
and conditions to effectively support the planned change. Subsequent research 
including the DESSI study by Crandall et al. (1982) demonstrated that "externally 
developed programs and external consultants can be extraordinarily effective in 
stimulating and supporting local efforts to improve practice" (p. 14). 

Policy goals are relevant to classroom teachers. Another misconception from the 
Change Agent Study was the assumption that the policy structure, i.e., federal, state, 
and local policies, was relevant for classroom teachers who implemented the policy 
into classroom practice. In reality, the structures of greatest importance to teachers 
may have little or nothing to do with specific policy goals or objectives. Professional 
associations, colleagues, or various other factors are of greater relevance to the daily 
concerns of most teachers. 

As the above discussion shows, our understanding of the nature and process of educational 
change continues to evolve. Fullan (1991) appropriately stated, "The reader who by now has 
concluded that the theory of educational change is a theory of unanswerable questions will 
not be too far off the mark" (p. 110). Change poses many dilemmas with few clear answers 
on what to do. We turn now to one of the more recent ideas of educational change, the idea 
of change as systemic reform. 



E. The School as the Organizational Focus of Change 

Within this new conceptual paradigm of the systemic nature of educational change, there 
is an increasing emphasis on the characteristics of the school as an organizational system. 
Researchers are beginning to look at schools as organizations and to draw lessons from the 
literature on successful organizations and organizational change. School-wide reform efforts 
also are structured to address the school as an organizational unit. For example, within the 
Coalition of Essential Schools, school restructuring efforts are understood as an 
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"organizational learning process" involving the substance of change, the process of change, 
and "organizational understandings" (Prestine and Bowen, 1993). 

A review of the research on school factors and their relation to student achievement 
demonstrates this shift toward a more school-wide, organizational system focus. Research 
conducted prior to the early 1970s examined variables such as school facilities, size, or 
resources which overall produced only weak or inconsistent relationships with student 
outcomes (Anderson, 1982; Centra and Potter, 1980; Coleman et al., 1966; Mayeske et al., 
1972). At about the same time, a variety of educational innovations were being developed 
and adopted without clear goals or understanding of the change being adopted. Since little 
thought or effort had been devoted to the actual implementation and follow-through of these 
innovations, most failed (Fullan, 1991). Thus, there was an abundance of pessimism in 
educational research. 

Subsequently, in the late 1970s and early 1980s, important reviews of research on teaching 
and teacher effectiveness (e.g., Dunkin and Biddle, 1974; Rosenshine and Furst, 1973) began 
to indicate more promising results of change efforts. There was also the beginning of major 
research efforts to improve the effectiveness of schools. Although much of this research was 
empirical and generally not guided by specific theories (Brophy, 1979), a considerable 
knowledge base began to develop. 

For example, research on specific teacher behaviors, such as "direct instruction" (Rosenshine, 
1979) and "active teaching" (Good and Grouws, 1975; Good et al., 1983) were associated with 
higher levels of student achievement. Other work showed the relationship of principal 
leadership, and principal involvement in instruction (Brookover et al., 1979; Edmonds, 1979; 
Ellett & Walberg, 1979; Rowan et al., 1983), and administration-faculty rapport with student 
achievement (Bell, 1979; Ellett and Walberg, 1979). School climate variables, such as good 
teacher-teacher relationships, and good student-teacher relationships (Wynne, 1980), were 
linked with higher achievement outcomes, as were variables such as values and goals of 
persons and groups within the school. For example, a "constant press for excellence" 
(Brookover and Lezotte, 1979), and agreement on academic goals, with incentives and praise 
for achievement, were linked with student success (Wynne, 1980). As more recent examples, 
school reform efforts such as the School Development Program (Comer, 1993-94) and the 
Coalition of Essential Schools (Coalition of Essential Schools, 1989) focus on approaches that 
require commitment to goals and integrated efforts from all participants within a school 
In reform models such as these, there is a clear awareness that involving only certain 
segments of the school in change will most likely lead to failure in the effort, and that there 
needs to be a pervasive commitment within a school to the stated goals and principles. 
These models recognize the need to work with schools as key units and to work with all 
levels within the school as an organizational system. 

1. Organizational Perspectives in the Study of School Change 

The social-organizational approach requires a broader, more complex perspective on 
the nature of the school. Within this approach, it is important to examine the 
interactions among levels within the school when addressing some of the traditional 
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questions that have been asked of schools. Bidwell and Kasarda (1980) used a socio- 
organizational approach to reexamine the earlier school-effects studies that had 
shown no significant effects of school-level variables on student outcomes. 

In this reexamination, the authors argued that a major conceptual and design error 
was present in much of the earlier research. They attributed the previous negative 
findings to the failure of that research to examine more closely the linkage between 
school factors (e.g., the school resources actually received by students) and student 
outcomes. For example, within the socio-organizational approach, the effects of 
resources on outcomes are examined in relation to the instructional units being 
studied, i.e., resources are examined at the level of the schooling recieved by specific 
students. This approach is more appropriate since resources within a school often 
may not be evenly distributed. Where resource distribution varies for different 
instructional units and thus for students, evaluation of the effects of resources should 
be examined at the level of the schooling (defined as the process through which 
instruction occurs, social organization of classes, curriculum, and other factors) 
received by the students. Bidwell and Kasarda demonstrated that when outcomes 
are examined in relation to resources at equivalent levels, then effects are found. 

Bidwell and Kasarda's analyses, and the work of other researchers within the 
organizational systems perspective (e.g., Rowan, 1990; Raudenbush and Bryk, 1989) 
provide a useful and promising approach to the study of change within schools. 

2. Two Contrasting Models of School Organization for Change 

The organizational perspective on school change essentially requires a revision of tl... 
common views of schools, just as school-wide change requires basic revisions of how 
schools function. Schools as they have generally operated in the past— and as most 
continue to operate— have been described as "loosely-coupled systems." That is, 
schools are described as systems without tight bureaucratic controls over teaching, 
and in which there is considerable autonomy and isolation of teachers within their 
classrooms (e.g., Goodlad, 1984). As a consequence, schools also have been 
characterized as lacking the types of organizational supports that would facilitate the 
needed professionalization of teaching (Rowan, 1990). 

Rowan (1990) points out that there are now two waves of reform: efforts toward 
increased standards (intensification) and efforts toward increased professionalization. 
The first wave would tighten bureaucratic controls; the second would increase 
professional controls. Schools in which either of these types of changes succeed will 
no longer be characterized as loosely coupled systems. Thus, new organizational 
models of schools must be developed to examine change. Also, research on changes 
in teaching, learning, school management practices, and other change efforts will 
need to take into account organizational system factors. It will be important to 
understand how organizational style affects teaching and learning and, ultimately, 
student outcomes. 



16 



9 

ERJC 



30 



As a starting point, Rowan (1990) defines two separate models of school organization 
that parallel the differences in the reform waves: "control" and "commitment." The 
two models represent very different modes of operation for teaching and school 
processes. They also represent different assumptions about the change process. 

Control model. The control model relies on strong bureaucratic control and a 
complex system of input, behavior, and output controls. The objective is to increase 
student achievement by requiring specific teaching processes and by standardizing 
student opportunities for learning. 

Commitment model. The commitment model focuses on the enthusiasm and 
professionalism of committed teachers, on collaboration among teachers, and on 
teacher participation in school decision-making. This has also been defined as an 
"organic" management model and is one in which bureaucratic controls are loosened 
rather than strengthened in order to increase practices that will ultimately improve 
student outcomes (Rowan, 1990, p. 354). 

Neither the control model nor the commitment model is likely to exist in its pure 
form; however, by recognizing these models, we can better interpret and understand 
interactions among specific change efforts within an individual school. Also, in 
discussions of change processes, it is helpful to recognize that differences in 
viewpoints about change may be due to these underlying differences in assumptions 
about the model for change, that is, assumptions about how change within schools 
should proceed. 

3. Implications of an Organizational Perspective for Studying School Change 

An organizational perspective requires that a study of the school incorporate 
observation of the school's operation at a number of different levels, rather than 
focusing on only one, and that examination of outcomes of specific change efforts be 
carried out at levels appropriate to the focus of the change effort. Taking this 
approach can help to clarify the processes that produce greater or lesser levels of 
success in implementation of specific change efforts. As a result of this better 
understanding, we would expect to obtain better guidance for policy ar.d practice. 

Through an organizational perspective, one may be able to better understand why 
certain change efforts may succeed in some schools and not in others. For example, 
Rowan suggests that classroom level factors, such as the teachers' perceptions of the 
nature of their work in implementing a particular instructional approach, can affect 
the success of change activities in other areas of the school, such as specific 
management level changes. In cases where teachers perceive their work within 
classrooms as non-routine and complex, they will view collaboration and professional 
exchange as helpful; in cases where teachers view their work as routine and 
straightforward, they are less likely to collaborate. Teachers implementing new 
classroom practices with substantial ongoing decision-making to adjust to individual 
student performance are likely to perceive this as complex work and are likely to 
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perceive sharing with others as a helpful and meaningful resource. In this case, 
change efforts that promote collaborative activities will probably take hold more 
readily. 

However, if new instructional approaches are implemented that teachers perceive as 
straightforward and less complex, then collaborative sharing with others is less likely 
to be perceived as important to them. The probable result is that mechanisms for 
teacher sharing of instructional practices will not take hold. Thus, viewing school 
change through an organizational perspective that incorporates the various levels or 
areas of focus within the whole school can open the door to more clarity in 
understanding factors that affect the success of change efforts. 

McCaslin and Good (1992) make a similar point with regard to instructional activities. 
They argue that it is incongruent to expect students to fit into a "risk-taking," more 
student-directed, approach to 'instructional activities when classroom management 
systems are more teacher-directed and closed to student participation. In other 
words, more "organic" or "commitment" models in terms of student participation are 
not likely to coexist comfortably within a "control" approach to classroom/ school 
management. Thus, the examination of interactions between tasks within the 
classroom and the management form in use may provide some important 
understanding of factors related to successful implementation of change efforts. 

F. The Context for School Change 

In this chapter, we have described some of the more prominent conceptual models and 
perspectives that have guided the study and practice of educational change efforts, and how 
these assumptions have altered over the years. Whereas in the 1970s, educational change 
efforts were concentrated on small-scale, piecemeal innovations, the more recent emphasis 
has been on the implementation of sweeping, systemic reform. The reform efforts are being 
carried out at the national, state, and local levels. 

One important context for any school is the district in which it is located. Many districts 
have been taking major steps toward district-wide reform and restructuring, with 
implications for schools in terms of the level and nature of the support provided for change. 
In addition, the nature and process of change within districts may provide important 
parallels for change within schools. Districts that are planning substantial reform efforts 
engage in "strategic planning" for change, involving setting goals and focuses for change, 
and identifying a process for change. Thus, the components of strategic planning for change 
within districts, as described in the next chapter, are another source of key background 
information for the study of school change involving limited English proficient students. 
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CHAPTER 3 



STRATEGIC PLANNING FOR CHANGE WITIN DISTRICTS 



A. Introduction 

Strategic planning, emphasizing long-term goals and system-wide improvement, has long 
been an integral part of the business world. The notion of strategic planning for educational 
change, however, is a relatively new one, with the bulk of the research reported within the 
last ten years. As the scope of educational change and restructuring efforts widens, 
educators concerned with the process of change have come to regard strategic planning as 
the only sound basis for designing a new educational system. Strategic planning is a means 
of first developing a vision of what the future system should look like, and then working 
to make that vision a reality. Strategic planning, when successfully applied to the 
educational context, meets both the educational and social needs of the community, 
providing community members with a sense of ownership in the educational system 
(Herman and Kaufman, 1991). 

In this chapter, findings are presented from selected research related to strategic planning 
for educational change. Primarily, research on strategic planning in districts serving K-12 
has been included. Findings are also incorporated from a recent focus group report on 
master planning in school districts serving limited English proficient students (Felsen, 
Hopstock, and DiCerbo, 1994). The focus group consisted of ten superintendents from 
across the United States whose districts were developing or implementing district-wide 
master plans. Each district submitted a copy of the master plan that had been designed. 



B. Relationship of Strategic Planning to the Change Process 

Research on educational change efforts suggests that no one theoretical model underlies 
strategic planning in the educational context. To a large extent, beliefs about change and 
the process by which change is accomplished determine beliefs about strategic planning, and 
strategic planning models parallel change models. One perspective suggests that change is 
relatively linear and orderly, a series of sequential steps that begins with a comprehensive 
strategic plan. Change is thus a virtually guaranteed consequence of strategic planning, and 
the planning phase is separate from subsequent stages. Loucks-Horsley and Cox (1984) 
seem to hold this perspective, describing the change process in terms of three discrete 
phases — initiation, implementation, and institutionalization — and suggesting that "each phase 
requires different actions, activities, and arrangements" (p. 6). Planning is part of the 
initiation phase. 

As educational reform efforts become more numerous and the nature of educational change 
is better understood, there is an emerging sense that change is a much more disorderly 
process than previously believed. Fullan (1991), for example, warns that "change is not a 
fully predictable process" (p. 107), a sentiment echoed by Conley (1993a), who notes that 



"careful, predictable, staged strategies for change" may not be of much value (p. 333). In a 
similar vein, Tyack and Tobin (1994) argue, "Almost any blueprint for basic reform will be 
altered during implementation...." (p. 478). The kiea here is that, no matter how well- 
conceived and thorough in detail the plan, it will not be translated exactly as envisioned. 
Tyack and Tobin suspect that this alteration may be a virtue, a way of adapting blanket 
reform to local needs (see also Louis and Miles, 1990; Carlson and Awkerman, 1991; and 
Herman and Kaufman, 1991). 

The idea that change cannot be readily predicted, that change is, in fact, "chaotic," (Fullan 
and Miles, 1992, p. 749) is reflected in the research on strategic planning. Strategic planning 
is more and more viewed as a means of continuous improvement of the educational system. 
A number of authors subscribe to this view, including Cook (1990); Bollin and Eadie (1991); 
Herman and Kaufman (1991); and Tyack and Tobin (1994). 

For example, Herman and Kaufman argue that strategic planning is not simply a tool for 
developing a written planning document but rather an ongoing process for serving societal 
needs. Bollin and Eadie (1991) take this same perspective when they discuss what they term 
strategic management, a combination of strategic planning and the continuing process of 
institutionalizing the plan. Whereas strategic planning is static, strategic management is a 
dynamic process. Louis and Miles (1990) describe evolutionary planning, a much longer 
process than strategic planning that calls for continuous review and refinement. Louis and 
Miles liken evolutionary planning to "taking a journey. There is a general destination, but 
many twists and turns as unexpected events occur along the way" (p. 193). 

A second theoretical issue with implications for strategic planning is the extent to which 
change is brought about through a political, i.e., top-down, process or through a consensual, 
bottom-up process. Authors such as Loucks-Horsley and Cox (1984) state that everyone 
involved in the school and community has a role in change efforts, and few would disagree 
with this conclusion. Where theoretical views differ is not in terms of who is involved, but 
when, how, and how much. Cook (1988; as cited in Hayden, 1993) believes that strategic 
planning needs to involve everyone at the top and the bottom of the organization 
simultaneously. Fullan (1991), for his part, suggests that strategic planning includes top- 
down initiative but bottom-up participation. This latter notion — that initiation of the process 
comes from a few planners at the top of the organization — is shared by many authors. 

Comments from district superintendents (Felsen, Hopstock, and DiCerbo, 1994) indicate that 
the strategic planning that currently takes place in school districts includes both political and 
consensual elements. The relative combination of each is dependent, at least in part, on the 
management structure of the district. Thus, while district staff are the main initiators and 
coaches in the strategic planning process, they play a more prominent role in districts with 
centralized management structures and/or where strategic planning is mandated by the 
state. Where site-based management structures are in place, district staff tend to have less 
involvement. LeBuffe (1993) described an analogous situation when differentiating between 
two types of planning: strategic planning and campus planning. Strategic planning is 
defined as a top-down, district-wide process that involves relatively few people and takes 
place over a relatively short period. 
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Campus planning, in contrast, is a bottom-up, campus-wide, long-term process involving 
a relatively large number of people. Findings from Conley (1993b) suggest that, in practice, 
school district planners do not make distinctions between the political and consensual, and 
the planning process tends to contain elements of both simultaneously. Conley also refers 
to technicist elements of planning — elements such as plan implementation that require 
adherence to detail but not necessarily broad participation. 

C. Pivotal Steps to Successful Strategic Planning 

Prevalent among the research is the idea that the change process includes a number of 
interrelated steps or phases. The strategic planning process is typically described as one step 
or set of steps in the process toward change. Conley (1993a), for example, seems to be 
discussing strategic planning when he describes "vision building" — creating shared 
agreement on values, beliefs, purposes, and goals that is "clear enough to enable participants 
to make choices that help move the organization toward achievement of the general values, 
beliefs, purposes or goals" (p. 346). Conley lists the following four phases of vision building: 

■ A readiness phase in which people are made aware of the issues, commit to 
change, and organize for change. 

■ A data collection phase that includes the collection of data and the creation 
of a database. 

■ A vision-building phase that includes the development of a mission 
statement and the setting of priorities. 

■ An action planning and implementation phase including implementation, 
monitoring of progress, modification or revision, and evaluation. 

The results of the vision-building process described by Conley is a vision statement or 
blueprint for change that sets the stage for action. Elements of the four-step process are 
contained in models of strategic planning described by other authors. Fullan (1991), for 
example, asserts that "above all, planning must consider the pre-implementation issues of 
where and how to start, and what readiness conditions might be essential prior to 
commencing" (p. 111). Louis and Miles (1990) discuss something similar to a readiness 
stage when they talk of "effective action" as a way to get people ready for planning. The 
two also describe vision building or the development of inspirational themes, i.e., "dynamic 
shorthand descriptions of the change program" (p. 212). 

The Cambridge process of strategic planning, a process advocated by Bill Cook, specifies 11 
steps: (1) communicate about the need for planning; (2) collect important data about the 
organization prior to planning; (3) select a strategic planning team of 25 people; (4) conduct 
the first planning session, which develops the planning components; (5) communicate to 
stakeholder groups the draft of the strategic plan; (6) select action planning teams to develop 
action plans to implement each identified strategy; (7) conduct the second planning session, 
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which considers each action plan; (8) prepare a summary of the final plan; (9) present the 
plan to the board for consideration and approval; (10) implement the plan; and (11) update 
the plan annually (Hayden, 1993). 

Herman and Kaufman (1991) developed the following 12-step strategic planning model: (1) 
choose the primary client; (2) identify the vision; (3) identify beliefs and values; (4) identify 
current missions; (5) identify needs; (6) identify matches and mismatches; (7) reconcile 
differences; (8) reaffirm the ultimate goal; (9) identify strengths, weaknesses, opportunities, 
and threats (SWOTS); (10) create decision rules; (11) develop strategic action plans; and (12) 
put the strategic plan to work. Kaufman and Herman later reduced their list to nine steps, 
but the basic model remained the same (Rinehart, 1992). 

Other authors, including Bollin and Eadie (1991) and Dlugosh (1993) present comparable 
models. The models that have been developed, although different in terms of level of 
specificity, have a number of common elements. For example, in each, strategic planning 
begins long before anything is put on paper and continues long after. Each model advocates 
preparation for planning through data collection and delineation of purpose; the creation 
and implementation of specific action plans; and evaluation and revision on an ongoing 
basis. Both Dlugosh (1993) and Cook (1988; as cited in Hayden, 1993) propose a strategic 
planning team of approximately 25 people. Like Dlugosh (1993) and Herman and Kaufman 
(1991), Bollin and Eadie refer to the identification of strengths, weaknesses, opportunities, 
and threats (SWOTS) as a necessary part of the strategic planning process. 

There are some important differences among the models. For example, while Conley 
acknowledges that "community members should be involved in the process of vision 
building" (1993a, p. 317) and the Cambridge process includes the need for all stakeholders, 
i.e., those influenced by the plan, to be represented on the strategic planning team, Herman 
and Kaufman propose their model only for "school board members, administrators, and 
other planners" (p. 25). Herman and Kaufman suggest, however, that planners choose the 
community and society as their primary clients and keep their needs in mind when 
planning. In each of these models of strategic planning, initiation of the process stems from 
the efforts of a relatively small number of people. Correspondingly, one of the conclusions 
reached by Louis and Miles (1990) is that successful planning efforts typically begin with a 
small group of interested participants under the sponsorship of the principal or other 
legitimate leader. 

Findings from ten focus group districts (Felsen, Hopstock, and DiCerbo, 1994) indicate that 
the process of strategic planning followed a similar pattern, with each district including the 
following elements: 

■ Promote a climate of change. 

■ Obtain widespread participation. 

■ Anticipate resistance. 
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Delineate plan goals, objectives, and strategies. 



■ Monitor plan implementation progress. 

■ Allow flexibility to adjust/revise plans. 

1. Promote a Climate of Change 

According to focus group participants, many districts start the process of strategic 
planning by setting up the conditions for change — collecting, summarizing, and 
disseminating information about the points that the plan is expected to address, e.g., 
the extent of the dropout rate in the district and effective methods of instruction for 
LEP students. This type of data dissemination has improved community awareness 
about important concerns and has increased support for new programs. 

2. Obtain Widespread Participation 

Focus group participants invariably agreed that participation in the planning process 
must be extensive — district and school staff as well as community members should 
all be involved — because the more people who ,r buy into" the plan at the beginning, 
the easier it is to implement later. 

3. Anticipate Resistance 

Related to promoting a climate of change and building support is the expectation of 
at least a certain amount of resistance to planned change. Resistance was viewed 
among the focus group participants as a natural part of the strategic planning 
process, and two main strategies are used to overcome it: reteaching and replacing. 
The first strategy assumes that key stakeholders in the process can be persuaded to 
support the strategic plan if they are given enough information to critically assess its 
merits. The second strategy is essentially negative reinforcement, i.e., those who do 
not support the strategic plan are fired or ignored. 

4. Delineate Plan Goals, Objectives, Strategies 

Making decisions on what to include in the plan is often a back-and-forth process 
between district and school staff and community members — a process that was 
referred to as playing an accordion — as plans are drafted, reviewed, and revised. 
Although many districts are moving toward site-based planning, district 
superintendents and assistant superintendents are often instrumental in setting a 
schedule for the completion of tasks, assigning responsibilities, and providing 
resources and assistance for plan development. 
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5. Monitor Plan Implementation Progress 

District superintendents emphasized the importance of periodically reviewing the 
implementation of the plan. What objectives have been accomplished? What 
remains to be done? What have been the outcomes so far? 

6. Allow Flexibility to Adjust/Revise Plans 

Most focus group districts understand the value of revising unrealistic goals or goals 
that, upon reflection, do not meet community needs. Focus group participants 
generally agreed that the strategic plan is not a static document. 



D. Success and Failure with Strategic Planning 

Educational change efforts, despite good intentions, often fail to achieve substantial or long- 
lasting change (Fullan, 1985;1991). Fullan lists a number of potential obstacles to change, 
including ignorance on the part of educational leaders as to how the change process works, 
and the complex and unpredictable nature of change that may stall even good efforts. 
Nonetheless, Fullan argues that deliberate or planned changed is possible through effective 
planning, i.e., planning that involves continuously acting, reflecting, and revising— the 
evolutionary process described by Louis and Miles (1990). Louis and Miles suggest that 
many of the weaknesses inherent in the planning process can be circumvented by this 
evolutionary planning process, which takes advantage of incremental successes to build up 
certainty in the change process and thus support for change. They argue that support is 
especially important when there is a great deal of segmentation between different 
departments, classrooms, etc. Since opposition is diffuse, small-scale planning efforts may 
be easily implemented without substantial support; however, more profound change is 
impeded by the lack of communication. Conley (1993a) discusses a similar problem in 
strategic planning, i.e., fragmentation or the failure to link the stages of vision building and 
goal setting with the nitty-gritty detail of putting plans into action. Other authors, including 
Kaufman and Herman (April, 1991), list several mistakes to avoid when planning, e.g., 
setting narrow goals and unclear objectives and failing to include a representative group of 
planners. The majority of these lists are simply the reverse of what should bs included in 
the strategic planning process, i.e., the pivotal steps. 

Unfortunately, since strategic planning for education is a relatively new concept, much of 
the research is theoretical rather than practical. Findings are thus limited, and it is difficult 
to gauge the results of plans only recently implemented. Nevertheless, a growing body of 
research studies addresses the application of strategic planning in schools and school 
districts. 

A recent article by Mirel (1994) offers some insights into why educational change efforts may 
fail. His findings, although targeted at the educational change process in general, have some 
relevance to strategic planning. Mirel analyzes the failure of a $1.25 million educational 
reform effort begun in 1992, the Bensenville New American School Project. It called for 
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profound educational change, including using nonschool sites in the community as 
educational settings; keeping schools open and available seven days a week for both 
children and adults; grouping students by ability and interest; and focusing on authentic 
assessment measures of achievement. On the basis of his analysis, Mirel makes several 
conclusions about educational change. He argues, for example, that gearing reform to local 
needs and culture and involving key stakeholders in the process are not enough to 
guarantee successful change. It is also important to consider the role of the community and 
teachers in the educational change process and to more clearly address their political and 
economic concerns, e.g., the possibility of increased taxation, and threats to job security that 
may result from the reform. Moreover, Mirel stresses that "in educational reform, perhaps 
less is more" (p. 517). When significant change is proposed, it is extremely difficult to 
achieve widespread endorsement. Yet, endorsement by parents and other stakeholders is 
necessary before meaningful change can occur. Mirel does not suggest that sweeping 
change is never possible or desirable, but he does note that less radical changes have been 
successfully introduced in the same community. 

Rinehart (1992) noted some positive effects of strategic planning efforts in a school district 
in the southeastern United States. The district comprised approximately 3,600 students 
representing 30 different countries and 15 different languages. The strategic planning 
process included: choosing groups of stakeholders (university, business /professional, 
governmental, ministerial, parents, retired citizens, students and alumni, support staff, and 
teachers); presenting an Ideas Seminar regarding the future of schooling, the economy, 
technology, and technical education; identifying strengths, weaknesses, opportunities, threats 
and strategic issues needed for implementation of their vision; electing two representatives 
to serve on a Local School System Strategic Planning Committee and to develop the strategic 
plan; and holding a town meeting to present the plan to the community for review. 

Rinehart did not identify quantifiable outcomes associated with the strategic plan, but 
benefits of the planning process included the identification of goals, objectives, and 
strategies. The primary limitation to the process was failure to include a truly representative 
group of participants, i.e., while there was a mix of participants, they did not match the 
demographics of the community. In addition, principals did not have input into the vision- 
building stage of the plan and this ultimately limited their participation. 

Nebgen (1991) describes some concrete results from the strategic planning efforts of one 
school district. The strategic planning process, based on the process advocated by Cook, 
was associated with improved student achievement levels, decreased violence and substance 
abuse by students, and increased community business outreach efforts. Nebgen found that 
most problems along the way came from the failure to communicate with staff and 
community during the planning process. 

Findings from the focus group districts suggest that strategic planning is sabotaged by 
resistance to change and by bad plans. 
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1. Resistance to Change 



As discussed earlier, focus group participants stressed the importance of having 
strong community and school staff support during the planning process. Those 
resistant to change can often be persuaded with data that illustrate the need for 
change or, in some cases, resisters can be replaced or ignored. 

2. Bad Plans 

Plans may be bad for a number of the reasons discussed above. Plans may be 
developed by a small group of individuals without the input of significant 
stakeholders. Plans may be inconsistent with district management style, e.g., a highly 
centralized district trying to develop a site-based plan. Plans may contain unclear 
or unrealistic goals (goals may be unrealistic because they cannot be achieved, or 
because they cannot be achieved with current resources). Plans may not include 
strategies for the achievement of the goals that have been set. Plans may be inflexible 
and therefore more likely to be thrown out. 

Overall, findings from the research suggest that, whatever the actual outcomes, people feel 
good about the process of strategic planning. The process of strategic planning, when it 
opens up communication lines and diversifies decision-making, seems to have positive 
consequences beyond the outcomes specified in the planning document. 

E, Implications of Strategic Planning Efforts for Limited English Proficient Students 

The bulk of the literature on strategic planning does not address the specific concerns of 
limited English proficient students and how these are accommodated within the strategic 
planning process. Nevertheless, findings from a case study of strategic planning in a 
multicultural district (Rinehart, 1992) suggest that strategic planning has advantages over 
other types of planning — advantages that may have a positive influence on the learning 
environment for LEP students. Strategic planning can help achieve the goals of a 
multicultural student population because it considers future possibilities, typically embraces 
representative stakeholders, and includes the realization that a variety of factors influence 
learning. 

Rinehart argues, however, that there is a danger that when strategic planning is only in the 
hands of individuals from the dominant culture, the planning document may not be 
representative of the needs of multicultural populations. Strategic planning for multicultural 
populations should address five main goals: equal opportunity to learn; positive attitudes 
toward other students; improvement in decision making and social skills; a cross-cultural 
perspective; and recognition of the value of cultural differences (Rinehart, 1992). As 
Rinehart notes, these goals are worthwhile for all students, and at least one focus group 
participant remarked that a well-conceived strategic plan addresses the needs of every 
student in the district. In fact, the strategic planning documents reviewed by Conley (1993b) 
and by Felsen, Hopstock, and DiCerbo (1994) include very similar goals. 
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Findings from focus group participants indicate that the way in which districts approach 
strategic planning for limited English proficient students depends on the size and 
composition of the limited English proficient population, state legislation, and district goals. 
In districts with very large, homogenous limited English proficient populations, the general 
strategic plan is often directly focused on limited English proficient education, promoting 
such educational strategies as whole language reading and native language instruction. 
These districts may also develop a separate plan for those students in addition to the general 
plan. Districts with few limited English proficient students, or districts where those students 
are concentrated in a few schools, typically incorporate the needs of those students within 
the general plan and/or in a separate handbook of guidelines and procedures (Felsen, 
Hopstock, and DiCerbo, 1994). 



F, Summary 

This review of the literature related to strategic planning for educational change indicates 
a number of areas of general agreement. First, strategic planning has come to be viewed as 
an effective means of achieving "extraordinary purpose" (Cook, 1990, p.74), i.e., profound 
change. Second, strategic planning is typically defined in terms of a process of constant 
adaptation to local conditions and requirements. The research describes a basic set of steps 
that are necessary to the process, although the sequence of steps may differ substantially 
across school districts, and the extent to which such differences affect strategic planning 
efforts has not been defined. Finally, the research suggests that strategic planning is most 
successful when the goals that are set have been clearly communicated to, and sanctioned 
by, those whom the plan will affect. Many of these same principles can be expected to be 
relevant to school-wide change. 
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PART II 

UNDERSTANDING THE NATURE OF CHANGE 
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CHAPTER 4 

A FRAMEWORK FOR DESCRIBING THE CONTENT OF CHANGE 



A. Introduction 

Research and innovation efforts increasingly are focused on school-level factors as important 
sources of input to student outcomes. However, how school-level factors are examined, and 
how the relationship between school and classroom and other factors is viewed, has changed 
substantially over the decades. There has been a shift from a focus on discrete school effects 
or innovation efforts to approaches that emphasize more systemic views of the school. In 
this perspective the school is viewed as an organization and characteristics of the school are 
described at different organizational levels. Viewed in this way, interactions among factors 
are taken into account and these may occur at any level and across levels. 

The recent legislation reauthorizing the Elementary and Secondary Education Act as the 
Improving America's Schools Act (IASA) states an emphasis on systemic change that is 
consistent with current research on reform efforts. IASA's Title VII continues to provide 
support for bilingual education programs for limited English proficient students, but with 
a new focus on systemic change. The purpose of the programs is to ensure that limited 
English proficient students meet rigorous standards for academic performance expected of 
all K-12 students and are included within the development of broad reform efforts. 

Title VII reflects the systemic emphasis of the IASA legislation through the types of 
programs defined and through the requirements stated for the various programs. For 
example, Program Development and Implementation Grants establish new "comprehensive, 
coherent" programs. Comprehensive School Grants provide assistance in implementing 
"schoolwide bilingual education programs for reforming, restructuring, and upgrading all 
relevant programs..." (Section 7114). Systemwide Improvement Grants (Section 7115) are for 
districts that serve limited English proficient students. 

Developmtiit of a framework for describing change within schools that serve limited English 
proficient students must take into account the complexity of the school as a system. In this 
chapter, we attempt to do so through our definition of the content of change. Our definition 
incorporates both the goals of a change effort and the focuses of change within the school, 
viewed as an organizational system. 



B. The Goals of Change 

We identify goals as one key dimension of change efforts. Goals essentially define the 
standards which the school is attempting to meet, and specific change activities will be 
shaped by those goals. 
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The legislative goals for Title VII of the IASA represent important new visions of how 
programs for limited English proficient students should be integrated into the overall 
instructional services within a school or district. In the legislation, the school is viewed as 
a key unit of change and a comprehensive, coherent, and integrated program is seen as the 
goal of this change, i.e., comprehensive school programs. Although the legislation does not 
provide a specific definition of Comprehensive School programs, key goals can be identified 
on the basis of what is present in the legislation and what is present in the research 
literature on effective instruction for language minority students. Our review of these 
sources has resulted in identifying four key goals of instructional services for programs 
serving limited English proficient students. Therefore, our framework defines a 
comprehensive school program for limited English proficient students as one which fulfills 
the following four goals: 

■ Coherence in instructional experience. 

■ Challenge in instructional content, concepts, and skills. 

■ Community within the school and in linkages outside the school. 

■ Concern for the whole student. 

We expect these goals will be relevant for all students, but we also expect that they will be 
of particular significance for change efforts related to the instruction of limited English 
proficient students. Each of these four goals is described below. 

1. Coherence 

Coherence refers to consistency in the nature of the instructional experience of 
individual students across different settings. This coherence can be present in a 
number of ways. First, there is consistency in the level and content of instruction 
received by a student within a school year and in articulation of instruction across 
school years. Second, consistency refers to the instructional approach that is utilized 
across the different settings in which each student receives instruction. Third, 
consistency refers to the flow or structuring of instruction within a typical school day 
or week for any student, such that instruction in one area is not disrupted or cut 
short to provide instruction in a second area. 

2. Challenge 

Challenge refers to the nature of instructional content, concepts, and skills that are 
the focus of instructional activities. Current reform efforts emphasize the need to 
move away from rote, lower-order cognitive skills as the focus of instructional 
activities and toward instructional content and tasks that help students develop and 
use higher order cognitive skills and that help students constantly advance the level 
and depth of their knowledge. 
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3. Community 



Community refers to linkages within the school and to linkages outside the school. 
Within the school, community refers to interaction among students or teachers, and 
between students and teachers. Community also refers to important linkages and 
understandings between the school and the community outside the school walls, 
including parents, business, and others. Communication, i.e., opportunities for 
sharing perspectives and ideas, is a critical support for — and means of — building 
community. 

4. Concern 

Concern for the whole student refers to attention and support given to the needs of 
the student, including physical and emotional health. Concern is shown through 
linkages with social services, assistance to families, counseling with students, among 
other approaches that help ensure that students enter their classrooms ready to learn. 

These four goals can be defined as the broad objectives of comprehensive school programs 
for schools serving limited English proficient students. Specific change efforts would 
therefore include activities within the school system that are directed toward one or more 
of these goals. It should be noted that the goals of comprehensive school programs actually 
are not ends in themselves. In fact they are intermediate goals in that they are expected to 
provide conditions that will lead to positive student outcomes, that is, to high levels of 
student performance and achievement (Figure 4-1). 

C The Focus of Change 

The second component of our definition of the content of change are those aspects of the 
school as an organizational system that are selected as the focus of the change effort. The 
five areas of focus identified in the model are drawn from research on school and teacher 
effectiveness ar d student outcomes, and from the research on change. Below, five areas of 
the school as an organizational system are defined as areas of focus for specific change 
efforts. These are the following: 

■ Structure and characteristics of the school; 

■ Climate/culture; 

■ Management / governance ; 

■ Linkages with community; and, 

■ Student instructional experience. 

These five areas of focus essentially represent what we believe are significant categories of 
variables to be included within a study of change. These five areas of focus are likely to 
interact such that changes within one area of focus can affect the nature of other areas of 
focus within the school. To describe a change effort, we believe that it will be important and 
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helpful to identify the primary area of focus of the change effort, recognizing that such 
interactions can occur. 

1. Structure and Characteristics of the School 

Several relatively fixed school characteristics define the nature of the school. Many 
of these characteristics have implications for the implementation of change or, 
perhaps, for the specific types of change efforts that might be selected. Examples of 
such characteristics are the grade level served by the school, the size of the school, 
the resources available to the school, and the student population served. While many 
of these characteristics are generally fixed and less amenable to change, they can be 
addressed by change efforts (e.g., restructured grade levels, school-within-a school 
to build smaller school units). 

2. Climate/Culture 

Climate /culture represents a general psychological sense or tone present within the 
school. It includes specific beliefs about students and about instructional approaches 
that may be shared by teachers to form a culture of beliefs about students or 
subgroups of students and about the teachers' own work and professional experience 
within the school. Climate /culture also includes the beliefs of students about their 
own instruction, their potential for success, and their own experience within the 
schools. 

3. Management/Governance 

Management/governance refers to the manner in which decisions are made within 
the school and to the participants in the decision-making. Collaboration among 
teachers for the purposes of sharing information about students and instruction and 
about school goals is also included here as an important component describing the 
school system. 

4. Linkages with Parents/Community 

Linkages with parents /community refer to linkages established with resources 
outside of the school. Although parents are considered a primary resource, others 
that exist within the community can be equally important. For example, linkages 
with businesses, ethnic or cultural organizations, community youth organizations, 
social services agencies, etc., develop not only opportunities for specific resources that 
can benefit the school but also support for the school. 

5. Student Instructional Experience 

Ultimately, all changes within a school should benefit students through improving 
the students' instructional experience. Student instructional experience refers to a 
number of variables that describe the nature of the instruction received by the 
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Figure 4-2 

The Content of Change Efforts in Comprehensive School Programs 
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student. This includes the nature of the curriculum (e.g., content, level, and pace of 
its presentation, etc.), the nature of the instructional approach used in the classroom 
(e.g., native language use, small group instruction, the types of instructional tasks), 
the structure of the program received (e.g., departmentalized, pull-out, self-contained 
class), among others. A description of the instructional experience of students must 
take a student-level focus in identifying services received (actual sets of services 
received by individuals) rather than a school-level focus (programs or services offered 
by the school). 



D. Summary 

Our framework for defining the content of change, as represented in Figure 4-2 comprises 
four goals of change and five areas of focus of the change. In the next two chapters we 
discuss research and findings relevant to each of the two types of change components. 
Chapter 5 presents findings related to the goals of change. Chapter 6 presents findings on 
the areas of focus of change. 
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CHAPTER 5 
THE GOALS OF CHANGE 



A. Introduction 

To describe and track change, we need to be able to describe what it is that is changing. We 
have defined four goals of change for comprehensive school programs for limited English 
proficient students. In describing how a school might approach the definition of a change 
effort, the model suggests that initially the school would review the current status of the 
school in terms of achieving each of the four goals and identify those areas in which the 
school's performance should be improved. For example, coherence may be weak even 
though performance on the other three goals is fairly strong. Or, the school may recognize 
that efforts are needed with regard to two or more of the goals. The definition of the 
change effort would then involve determining which aspects of each goal will be selected 
as more specific objective(s). Similarly, a research study of change within schools would 
identify the goals of the change being carried out, and identify the more specific objectives 
that the school has defined for addressing that goal. 

Within the framework for the description of the content of change we have outlined four 
goals of Comprehensive School Programs: coherence, challenge, community, and concern. 
Our assumption is that ultimately a school would work toward achieving all four of these 
goals but that within any specific change effort, only one or a few of these might be 
identified as goals. The four goals are further discussed in the sections below. 



B. Coherence 

Coherence refers to consistency in the experience of individual students within the school. 
As defined earlier, coherence can be present in a number of ways. Coherence can first of 
all be viewed in terms of the nature and level of the content of instruction received by a 
student across settings. Within any individual school year when the student is instructed 
in different settings with different teachers, the level of instruction and nature of the 
curriculum may vaiy in significant ways. For example, Moll and Diaz (1987) examined the 
contrast in level of instruction received by the same students in classes in reading in Spanish 
and classes in reading in English. While in Spanish, students were responding to higher- 
order questions regarding the content of their reading; in English class, reading was focused 
on lower-level decoding skills. For example, in Spanish reading, the students were carrying 
out more complex tasks such as drawing inferences and making predictions based on the 
content of their reading; this was not occurring in their English reading classes. Moll and 
Diaz successfully introduced a new approach into the English classes that incorporated use 
of higher-order cognitive skills as were used in the Spanish reading classes. They 
demonstrated that these students could benefit from the more challenging instruction within 
the reading classes in the students' second language. 
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An important and second form of coherence refers to consistency in the instructional 
approach utilized across different classroom 1 settings. Instruction is likely to be most 
effective when students can enter each new setting knowing what is expected of them and 
understanding their role in interacting with the teacher and other students. Where there is 
this type of consistency, students do not need to remember different rules for participation 
in different settings because all share the same norms for teacher and student roles and 
interaction in instruction. For example, within a coherent program, a student who 
experiences an "active learning" instructional approach (Lathrop, Vincent, and Zehler, 1993) 
within a bilingual class would also be given opportunities for active participation within 
other classes in which he/she is instructed. 

A third type of consistency or coherence refers to the flow or structuring of instruction 
within a typical school day or week for any student. The students' schedule should be 
structured such that instruction in one area is not disrupted or cut short to provide 
instruction in a second area (as might be the case, for example, if a student leaves a reading 
class to attend a special English as a second language or Chapter 1 /Title I class). Students 
need to begin instructional tasks knowing that they can become involved and complete 
them; teachers need to be able to complete work with a student on activities to their planned 
end point. 

In research on effective instruction, variables related to what we have defined as "coherence" 
of services have been indicated as critical elements of effective schools and services for 
language minority students. For example, Carter and Chatfield (1986) reported several 
characteristics of effective schools for language minority students. They included among 
these the role of the bilingual program as an integral part of the school, and coordination 
between bilingual and non-bilingual curricula and materials. 

In sum, within a coherent program, the student's instructional experience within a school 
is characterized by some very basic types of security for the student: security in knowing 
that the instructional day will be predictable and consistent in terms of the nature of the 
instruction received, in terms of the rules for interaction and participation in carrying out 
academic activities, and in terms of knowing that instruction in any one area will not be 
disrupted to begin instruction in another setting. Of course, coherence in and of itself is not 
necessarily good; coherence or consistency in use of poor instructional approaches would 
not be a desired achievement for the school. However, if the school also provides an 
appropriately challenging experience for students, then coherence should serve to magnify 
good practices within the school. 



C. Challenge 

Challenge refers to the nature of instructional content, concepts, and skills that are the focus 
of instructional activities. Too often, the level of instruction for minority students, including 
language minority students, is at the level of lower-order, basic skills, and the curriculum 
offered does not provide opportunities for more advanced course work or for instruction 
focused on higher-order cognitive skills (e.g., Oakes, 1990; Minicucci and Olsen, 1992a,b). 
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Yet it is clear that students need an opportunity to develop higher-order skills and in-depth 
understanding of concepts, whether instruction is in the native language or in English. 

Also clear is that students' lack of full proficiency in English does not preclude the use of 
higher-order academic tasks within instruction that is presented to them in English. For 
example, the Moll and Diaz (1987) study described earlier also demonstrated that the 
Spanish language students could be challenged by and could benefit from instruction 
focused on higher-order skills in their English reading classes. Warren and Rosebery (1990) 
have provided a similar example of language minority students' successful participation in 
more challenging instruction within the area of science. 

D. Community 

Community refers to linkages within the school and to linkages outside the school. Tliere 
is a collegial and collaborative community within the school (including shared decision- 
making), a shared culture and community among students and staff, and integral linkages 
between the school community (teachers, administrators), parent community, and business 
and other community members. 

Within schools characterized as a strong community, there is a communication among 
teachers and other staff, especially among those who work with the same students, to share 
information on instruction and instructional approaches, to share problem-solving, and to 
share successes. The school is generally not characterized by strict divisions between 
programs or departments that serve the same students. For example, a bilingual program 
is not segregated but is well integrated within the rest of the school (Carter and Chatfield, 
1986). The school reaches out to include parents (U.S. Department of Education, 1994), 
including parents of language minority students (e.g., Violand-Sanchez, Sutten , and Ware, 
1991), and also brings in other community and business persons as resources and as further 
sources of support for students and for the school in general. For example, at a classroom 
level, community persons with expertise might contribute to instructional thematic units by 
visiting and presenting to a class (cf., Moll et al., 1990, "funds of knowledge"). 

Communication is an important support for the building of a strong school community. For 
example, among students and students with teachers, Garcia (1988, 1991) has noted that 
effective instruction of language minority students is characterized by a clear emphasis on 
communication. He notes that in classrooms judged to be effective, instruction involved 
students communicating with each other and with the teacher for most of the day. 



E. Concern 

Students are not able to learn effectively and to reach their highest level of potential if they 
enter their classrooms hungry, in poor health, or under other forms of stress. Language 
minority students are often students from low-income backgrounds and likely to need 
additional support. The goal of the school to support these students by increasing their 
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minority students are often students from low-income backgrounds and likely to need 
additional support. The goal of the school to support these students by increasing their 
readiness to learn is demonstrated by development of linkages with social services, 
assistance to families, counseling with students, among other approaches. It is also 
demonstrated by a consensus among the staff that the physical and emotional health of the 
students is important so that they assist in identifying students who appear to be in need 
of these types of supports. One example of schools that make the whole student their focus 
are those which incorporate social services within their campus, e.g., access to a health clinic 
or to other social services providers. 
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CHAPTER 6 



THE FOCUS OF CHANGE 



A. Introduction 

The description of a change effort not only involves a statement of the goals of the change 
but also requires definition of the focus of the change activities. In our framework for the 
content of change, we have defined five areas of focus within the school system (Chapter 
4, Section C). These five areas of focus have been identified on the basis of distinctions 
found within the literature on the nature of change activities. We believe that these 
distinctions represent useful means of summarizing and examining change efforts within 
schools. 

In this chapter, five areas of focus of change efforts are described. These areas of focus were 
identified on the basis of research on change, including research on "restructuring", as 
described in the first section below. Sections C through G present discussion of the five 
areas of focus we have identified: structure and characteristics of schools, climate /culture, 
management/ governance, linkages with parents /community, and student instructional 
experience. 

B. The Description of Change Within Research on Restructuring 

Particularly since the mid-80s a considerable amount of research and practice has been 
carried out under the banner of restructuring. The term restructuring has been used to 
signal that there is an overall, systematic change in the school and in the school's approach 
to educating its students. There have been various approaches to definition of restructuring 
that describe key elements or dimensions of restructuring. These are outlined below. 

1. Conley's Twelve Dimensions 

Conley (1991) has outlined twelve dimensions of restructuring, which are 
incorporated within three categories of variables: central variables, enabling variables, 
and supporting variables (See Figure 6-1). The central variables are those that are at 
the core of the school's instructional process. The central variables include learner 
outcomes that are defined as the goals of instruction, the curriculum and materials 
used to present the instruction, and the specific instructional approaches or 
techniques implemented. Assessment is the means by which the learner's progress 
toward the stated goals is measured. 

The enabling variables either are necessary to or aid in the activities or changes in the 
central variables. Conley believes that in many cases, schools actually focus on 
enabling variables more than on the central variables in order to ultimately affect the 
methods and content of teaching. Enabling variables are the learning environment 
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Figure 6-1 
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(which includes the teacher/learner relationship, the grouping of students, the use 
of experiences outside of the school for learning, etc.); technology (which provides 
opportunities to enhance the quality and quantity of learning ); school-community 
relationships (which includes the roles of parents and of others in the community; 
Conley also suggests that this includes school-social service agency linkages ); and 
time (which refers to the structure of instructional periods, of the school day or year). 

Supporting variables are described by Conley as generally addressing "the 
organizational conditions of teaching and schooling" (Conley, 1993a, p. 110). He 
defines four supporting variables: governance (which refers to the nature of 
decision-making within a school, e.g., site-base management, participatory decision- 
making); teacher leadership (which refers to new roles and responsibilities for 
teachers, e.g., as mentors, as team leaders or teacher-researchers); personnel structure 
(which refers to the type of staff present, e.g., professional certificated staff and 
classified staff and their roles); and working relationships (which refers to contractual 
relationships between teachers and administrators and boards of education.) 

2. Building Blocks of Restructuring 

Lieberman and Miller (1990) take a different approach to describing restructuring. 
In describing the underlying elements of restructuring, they focus first on defining 
two separate categories of restructuring approaches: those based on principles and 
those based on procedures. Procedures refer to structural or organizational steps that 
support or promote steps toward restructuring. For example, providing increased 
staff development or offering planning time for teachers is an organizational step that 
provides a structure for and support for teachers becoming more active participants 
in instruction /organization of the school. 

Principles are used to provide frameworks for restructuring. The use of principles 
allows for and even assumes individual variation in how the principles are translated 
into specific strategies and practices. This approach sets up core values or beliefs that 
guide the restructuring effort and that are constant across different specific sites. 

Lieberman and Miller identify five key "building blocks" of restructuring: 

(1) Rethinking curricular and instructional efforts in order to promote quality and 
equality for all students. Lieberman and Miller describe this as the 
cornerstone of restructuring; it requires a willingness and readiness to 
question existing practices and to make modifications. 

(2) Rethinking the structure of the school. This building block derives from the 
assumption that if curricular /instructional changes are to occur, then these 
will require a change in how schools are organized and administered. Under 
this building block, Lieberman and Miller specifically refer to the role of the 
teacher, including teacher participation in school-based management and 
decision-making. 
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(3) Maintaining a two-pronged focus on a rich learning environment for students 
and on a professionally supportive work environment for adults. This 
building block cautions that attending to only one side of the teacher /school 
equation will not lead to effective implementation of change. 

(4) Recognizing the necessity for building partnerships and networks. These 
partnerships include relationships among schools involved in change efforts 
for the exchange of insights, practices, ideas. 

(5) Recognizing the increased and changing participation of parents and the 
community. Any restructuring process must include the meaningful 
involvement of parents and the community. 

Lieberman and Miller describe these building blocks as the "foundation for 
comprehensive change in our schools." For each building block, they see the 
necessity of a school defining a plan, a structure, and supports for implementation. 

3. Fullan's Three Dimensions of Change 

Fullan (1991) has described the multidimensional nature of change. He outlines three 
key dimensions: 

(1) Use of new or revised materials; 

(2) Use of new teaching approaches or strategies; and 

(3) Possible alteration of beliefs such as pedagogical assumptions or theories that 
underlie new programs. 

Fullan uses these dimensions to explain how the failure of an innovation to take root, 
or to be utilized effectively, can be better understood. For example, if the effective 
use of a specific instructional approach assumes a specific belief about learners, but 
the implementation involves persons who do not accept (or come to accept) these 
beliefs about the learner, then the approach is not likely to be sustained and /or not 
likely to be implemented effectively. 

4. Schools in Restructuring: Four General Areas of Change 

The Center on Organization and Restructuring of Schools conducts research on how 
schools are restructuring. Its objectives are to understand how restructuring actually 
is working and to identify just how comprehensive a phenomenon restructuring 
actually is. Its research has identified four general areas of restructuring (Newmann, 
1991; Prager, 1992): 

(1) Student experiences; 

(2) Professional life of teachers; 
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(3) School governance, and 

(4) Collaboration between schools and communities. 

Based on a survey of some 250 schools, the researchers found that few carry out 
comprehensive restructuring across all four areas; most schools reported implementing 
change efforts within the first two areas only (student experiences and professional life of 
teachers) (Center on Organization and Restructuring of Schools, 1992). There was little 
found in the way of attention to organizational changes in school management; the 
researchers suggest that changes in authority or coordination or social or political barriers 
might inhibit the more comprehensive structural overhaul implied by the last areas. 

These separate models of restructuring, although they differ in approach, identify several 
elements in common. We have based our selection of the five areas of focus of change 
efforts on key elements found across these four models and identified in other research on 
school change efforts. Research on the five areas of focus are described in the next sections. 

C. Structure and Characteristics of the School 

Prior to the early 1970s research on school factors and their relation to student achievement 
generally found few school-caused effects in comparison to the role played by student 
background and aptitude. The types of factors examined were most often variables such as 
school facilities, size, or resources, which overall produced only weak or inconsistent 
relationships with student outcomes (Anderson, 1982; Centra and Potter, 1980; Coleman et 
al., 1966; Vlayeske et al., 1972). However, other research challenged these findings. For 
example, research carried out in the years following the publication of the Coleman findings 
examined high -achieving or unusually effective schools and demonstrated that schools could 
have an effect (Mace-Matluck, 1987), Later research (e.g,, Bidwell and Kasarda, 1980) also 
revised some of the earlier negative conclusions with findings that specifically showed 
effects for school-level variables. 

While variables that describe the structure and characteristics of the school are often less 
amenable to change, it is still possible that they can become the focus of change efforts. 
When they are not themselves the focus of a change effort, they are nevertheless important 
aspects of the school to describe in any examination of a change effort, since school-level 
variables may interact with other variables that are directly addressed by a change effort. 
The following are some of the key variables that can be used to describe the structure and 
characteristics of the school. 

1. Level 

Much of the earlier research on effective schools focused on elementary level schools 
primarily (Mace-Matluck, 1987; Shann, 1990), However, there is a growing body of 
literature on the implementation of change efforts within secondary schools. This has 
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produced greater understanding of the differences between elementary and 
secondary schools. 

Organizational differences. Purkey and Smith (1985) showed that secondary schools 
are organizationally more complex than elementary schools in at least three ways: 
(1) they are typically larger, with more students, staff, and physical plant, and 
broader curriculum with a multiplicity of goals; (2) they are politically more 
complicated, with several administrative layers, such as department heads; and (3) 
they enroll students who are older, more developmentally advanced, with more 
diverse educational objectives and longer educational histories. 

While general principles of effectiveness can apply to both settings, the organizational 
differences for elementary versus secondary schools are significant since they may 
require adaptations in implementation of change and in research variables within 
studies of change. For example, in place of elementary-level emphasis on the 
principal as leader, department heads or assistant principals at the secondary level 
may be more important in setting a climate for success and in motivating for change 
(Hord, 1984). Thus, the principal in high schools may need to be an effective 
manager of the several "chefs" within the school to develop the type of "menu" for 
change that will be successful (Hord, 1984). 

Student differences. More so than elementary students, secondary students bring 
prior educational histories. These prior experiences can affect students' attitudes and 
norms for work and behavior. Students at the secondary level also have broader 
reference groups, beyond the home and family. These differences can mean that the 
students will be less passive and more resistant to change and that the change efforts 
will therefore need to plan for these factors. 

Structural differences. At the secondary level, students move through a 
departmentalized schedule, in which they move from teacher to teacher, class to 
class. This is in compar ^on to the elementary level where students are with one 
group and are instructed primarily by one teacher for the majority of the day. The 
secondary-level school most often does not provide opportunities for a student to 
"belong" to a specific smaller community as in the elementary grades. However, 
providing a more stable community in the secondary grades can be an important 
support for students, especially for recently arrived language minority students. 
Some change efforts at the secondary level address this need of students by placing 
them in groups who remain with the same teacher or set of teachers in a "family"-like 
unit across two or more years of school. Students can feel that they ,f belong" to a 
group, and teachers are able to see and appreciate change in students across years. 

2. Size 

Particularly in research that examines secondary schools, smaller institutions appear 
to have an advantage over larger institutions in creating a sense of community or 
shared norms and goals (Bryk, 1994). Larger schools are more likely to have more 
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complex administrative structures. They may comprise a broader range of student 
groups or tracks, all of which will have implications for how the school is structured 
organizationally and how it functions. These, in turn, will be possible targets of 
change efforts or may require adaptations of change efforts. For example, creating 
separate schools-within-a-school is one means of creafing the advantages of a smaller 
school within a school that is large. 

3. Resources 

Resources can be defined in terms of persons available to the school (including staff, 
students, parents, and others), materials, and time (Bidwell and Kasarda, 1980). 
These are discussed below. 

Persons as resources. Staff, students, parents, and other community members can 
all be included as school resources. In terms of staff, teachers' skills and. 
backgrounds serve as resources to the school as a whole (and can be distributed 
differentially within a school). Of special relevance to limited English proficient 
students would be teachers' knowledge, experience, training related to work with 
language minority students, teaching within bilingual programs, etc. Other variables 
include staff attendance (Torres, 1987), percent full-time staff and percent academic 
faculty (Louis, Marks, and Kruse, 1994). Specific change efforts might be directed 
toward increasing the skills resources available to the school through staff 
development programs; for example, staff development could be provided to 
heighten teacher awareness and understanding of the role of cultural differences in 
the classroom, to provide for understanding of language acquisition processes and 
language learning, to assist teachers in identifying students in need of social services. 

Students provide resources as well, in terms of language and cultural backgrounds 
and other experiences that they offer to the school and can share. Another "person" 
resource is parents where they take on roles and responsibilities within the school or 
provide services to support teachers or student instruction. In addition, other 
community members, including businesses, are resources to the extent that they bring 
skills and services into the school. If the goal of a change effort is to increase 
linkages with community organizations, for example, the result might be to increase 
resources in the form of special skills or services brought into the school. For schools 
serving language minority students, parents and other community members with 
language skills can be brought into the school to increase its language-related 
resources. 

Materials. Materials include books, maps, laboratory facilities, computers, etc. For 
schools serving language minority students, materials would also include those that 
are related to the specific language and cultural groups present. For example, books 
in Spanish available within the library create an important resource for the schooling 
of Spanish-language students, either for use in the classroom or for use by students 
and their families at home. 
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Change efforts may focus on the way in which materials are used within schools. 
For example, in order to rationalize and most effectively utilize existing resources, a 
school may make a concerted effort to break down walls between categorical 
programs through the pooling of resources (for example, allowing sharing of 
purchased books and materials across programs, or using joint funds to obtain new 
materials). This may be a significant effort, with possible implications for the 
development of a broader community in the school, especially when there are several 
different categorical programs within a school in which all have previously been kept 
strictly separate. 

Instructional resources/curricula for limited English proficient students. Access to 
a curriculum that is equivalent in content and level to that provided to English 
proficient students is an important issue for all schools. However, at the secondary 
level, access to instruction for limited English proficient students often presents a 
more serious problem than at the elementary level. Within secondary schools there 
is frequently a lack of access to courses that provide more advanced instruction 
within academic content areas (Minicucci and Olsen, 1992a,b; Oakes, 1990). To help 
address this problem, schools can offer advanced courses for students, in which 
instruction is adapted to the language needs of the students. (Doing so requires 
resources such as staffing and materials resources described above.) 

4. Time 

Time is also a significant resource and can be described in terms of the simple 
availability of time and in terms of structure of time. The amount of time available 
for an activity can refer to the length of the school day, time for academic subjects, 
time available to students for study within a specific academic area, or time available 
to teachers for joint planning. In fact, the availability of planning time for teachers 
is often viewed as a better incentive than monetary reimbursement (Richardson, 
1990), and time for meeting and talking has been described as a structural condition 
necessary for the building of a professional community within a school (Louis, 
Marks, and Kruse, 1994). 

The structure of time might also be included as a means of defining school resources. 
For example, flexibility in class scheduling allows the allocation of a longer period 
of time to a specific subject area or project; longer periods support in-depth work by 
students. Peyton et al. (1994), for example, discuss the importance of time as a 
resource to pull-out English as a Second Language teachers in implementing a 
specific instructional approach. They point out that insufficient time allotted can 
undermine and even preclude implementation of effective instructional activities. 
The effective use of time has been identified as a key to achieving success within the 
school (Goldman, Dunlap, Conley, 1991). 
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5. Student Aggregate Characteristics 



The student population served by a school is also a basic condition and resource 
within the organizational context. Students' age, ethnic and language background, 
level of English language proficiency, family socioeconomic status, and family 
educational background, among other variables, help to define the school. 
Additional student variables include mobility, percent limited English proficient, and 
percent language minority. These student characteristics have implications for what 
other types of resources are needed and, together with consideration of resources 
available, will be important elements in determining the nature of instructional 
programs structured within the school. 

The distribution of students to different instructional units within a school (as in 
tracking or in segregated special programs, such as special education or bilingual 
education classrooms) is one means of describing instructional experiences of 
students. Bidwell and Kasarda (1980) make a distinction between student composite 
attributes as resources helping to define the nature of schooling received by students, 
and attributes of individual students as student-level inputs. In essence, this 
approach acknowledges that the level of performance and other characteristics of 
students present in a class provide an important element of the instructional context 
to be described for any individual student. 

Any change efforts will need to consider what implications the student characteristics 
have for implementation of the change approach. For example, where there are 
language minority students present in the school, will all teachers be willing to 
collaborate and share responsibility or will a bias toward keeping these students in 
special and separate programs provide a serious barrier to whole school approaches? 
What types of special change-related activities and approaches or staff-development 
efforts will be needed given that many teachers will be working with language 
minority students? How can limited English proficient and fluent English students 
be given opportunities to work together? What are the best ways to accomplish this? 
These are the types of questions that change efforts will need to address once the 
nature of the student population to be served is recognized as a critical element in 
; any change. 

D. Climate/Culture 

Culture within organizations has been defined as a shared system of values, beliefs, and 
norms; as a "congruency of thought" and recurrent and predictable behaviors (Osborne, 
1993), According to Osborne (1993) educational culture is an extension of this corporate 
cultures into the educational setting. Within the effective schools literature, "total 
environmental quality" (Anderson, 1982) or climate (Brookover et al., 1978) is a key aspect 
linked to higher student achievement. Climate factors include positive expectations for the 
students, good administrator-staff and teacher-student rapport, and a general sense of 
community among students as well as staff (Anderson, 1982; Wynne, 1980). School climate 
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variables, such as good teacher-teacher relationships, and good student-teacher relationships 
(Wynne, 1980), were linked with higher achievement outcomes, as were variables such as 
values and goals of persons and groups within the school. For example, a "constant press 
for excellence" (Brookover and Lezotte, 1979) and agreement on academic goals, with 
incentives and praise for achievement, were linked with student success (Wynne, 1980). 
According to Deal (1990), a key element to successful reform is a renewal of commitment 
and a "transformation" of cultural values and mind-sets; Deal believes that these are aspects 
not addressed sufficiently by many change efforts. 

The most central dimensions of school climate have been described as "norms of the social 
system and expectations held for various members" (Brookover and Erikson, 1975, p. 364). 
High expectations for all students; a clear vision or set of goals, with a focus on instruction; 
and an orderly and safe environment have been among the characteristics identified as 
associated with effective schools (Mace-Matluck, 1987). These are included here as 
climate/culture variables. In addition, we include within climate /culture the presence of 
collaboration/collegiality among staff; the sense of community overall within the school 
among teachers, staff, students; social norms and values for students; teacher beliefs 
regarding students and their instruction; and an ethic of caring. 

1. High Expectations for All Students 

A supportive and effective climate /culture for students is one in which there is a 
"press for excellence" for all students consistent with the overall goal of challenge. 
Lezotte and Bancroft (1985) described effective schools as those in which there are 
high levels of achievement across all subgroups of students within the schools; i.e., 
schools in which there was both quality and equity. 

2. Clear Shared Vision 

The school as a whole, including students and teachers, should share the same vision 
in terms of goals and objectives and how these should be obtained. The extent to 
which students who see their school work as related to their future has been linked 
with growth in achievement (Ainley, 1994). This would be one indication of a sense 
of purpose and vision for the school at the level of students. School staff who share 
a common vision will be able to guide decision-making through that vision. 

3. Orderly and Safe Environment 

The presence of an orderly and safe environment has been noted as a characteristic 
of effective schools in several studies (Mace-Matluck, 1987). Related to this 
characteristic of the school climate overall, other researchers have noted (e.g., 
Lathrop, Vincent, and Zehler, 1993) that a predictable and "safe" environment in the 
classroom — one in which the students understand routines, rules, and procedures 
because these are clear, generally made known, and consistent—is also important. 
In particular, for language minority students, knowing what to expect and knowing 
the rules for participation provide an important basis that supports learning. 
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4. Collaboration/Collegiality 



An important aspect of school climate /culture is the extent of collaboration and 
collegiality among staff. This is promoted by communication and sharing among the 
staff and is often supported by a principal who provides support for teachers to meet 
and talk. Teachers who share respect for each other, who value each other's input, 
and who support one another build a sense of community that provides an important 
basis for school improvement efforts. 

A strong professional community within a school has been linked with positive 
effects on student engagement and achievement (Newmann, 1994). However, 
Newmann has noted that thus far such findings have been based on high school data 
only. (There will be research data available related to other levels in fall 1995.) 
Sharing and collegiality among staff members helps to sustain change (Simpson, 
1990); also, the culture of collegiality among staff is linked with 
management/governance variables, such as teacher participation in decision-making 
and working as a team. 

5. Social Norms and Values 

Prosocial behaviors, such as helping others, getting along and making friends across 
racial and ethnic groups, and showing respect are important school norms. Shann 
(1990), in a study of middle schools, noted significant differences among schools in 
the degree to which prosocial behaviors were present. However, Shann also noted 
that an ethic of caring (see below) was apparently necessary in conjunction with 
prosocial behaviors for there to be an effect on student achievement. 

6. Teacher Beliefs 

Addressing teacher beliefs can affect classroom practice, and individual teacher 
change, is affected by the overall climate and culture of beliefs within the school 
(Richardson, 1990). Also, teacher beliefs about their own efficacy can be affected by 
the outcomes of new practices and may interact with the specific types of 
instructional approaches used. 

Beliefs about the characteristics of students, about the nature of learning, and about 
the validity of different instructional approaches will affect teacher practices. Where 
teachers share similar understandings about instruction, they are likely to use similar 
practices in their classrooms. However, it is not always necessary to change beliefs 
first in order to introduce new practice (McLaughlin, 1990). Successful practice of 
innovations can lead to change in beliefs (Fullan, 1991; Richardson, 1990), thus 
changing the climate/culture within a school. 
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7. Ethic of Caring 



A concern with the development of a caring environment is one that deserves careful 
attention. According to Shann (1990), an "ethic of caring" promotes respect, 
affiliation, and commitment among adults and students and is a variable that may 
be especially important to at-risk students. McLaughlin (1994) attributes the effects 
of the restructuring efforts reported by Lee and Smith (1994) to the way in which 
those practices supported valued connections between students and adults within the 
school setting. 

The climate /culture can be changed through change efforts that address beliefs and 
understanding on the part of staff. For example, ongoing staff development for all teachers 
that provides information on language acquisition, on student diversity and its role in the 
classroom, and on effective practices in working with diverse students (e.g., Faltis, 1993) can 
play an important role in developing a school-wide climate or culture supportive of limited 
English proficient students. 

For schools serving limited English proficient students, a particularly relevant concern is 
whether there is one school climate or multiple climates among various sub-units of 
instructional services within the school, e.g., in special instructional programs for limited 
English proficient students versus regular instructional programs. In other words, it is 
important to be aware of the possibility of multiple layers or units within the school, each 
of which may have different sets of norms and expectations. This suggests an issue related 
to "coherence" of climate within instruction received by limited English proficient students. 



E. Management/Governance 

Research on effective schools identified principal leadership and principal involvement in 
instruction (Brookover et al., 1979; Edmonds, 1979; Ellett & Walberg, 1979; Rowan et al., 
1983) as consistently linked with student achievement (Bell, 1979; Ellett and Walberg, 1979). 
Within the more recent efforts to improve schools, the nature of school management, the 
forms of governance, and the type of leadership have also received considerable attention. 
In this more recent work, the focus is on decentralizing decision-making, relying more on 
site-based management, and increasing participation of staff in decision-making (e.g., 
Darling-Hammond, 1992; David, 1989; Goldman, Dunlap, and Conley, 1991). 

Two forms of management have been described as a means of summarizing distinctions 
found among schools. Rowan (1990) has pointed out a key distinction between "control" 
versus "organic commitment" (Rowan, 1990) management and decision-making strategies. 
A control strategy represents a top-down structure, in which administrators make decisions 
and provide specific directives and requirements to the teachers (e.g., a strong and directive 
principal). In an "organic" or "commitment" model there is a bottom-up structure, in which 
teachers are involved as professionals making decisions together with administrators. In this 
model, the principal is still a leader but does not lead in the traditional sense of the term; 
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instead, the principal becomes a "facilitative" (Goldman, Dunlap, and Conley, 1991) leader, 
a supporter of the networking and joint decision-making by teachers. 

The success of efforts to involve teachers in decision-making is linked with the nature of the 
decision-making: What areas of decision-making are opened to teachers and school staff? 
What resources do these decisions actually control? (Miles and Seashore-Louis, 1990; David, 
1989; Louis, Marks, and Kruse, 1994), Communication and decision-making among school 
staff will not become a sustained mode of operation unless some real purpose and effect can 
be observed. In examining teacher involvement in decision-making, it will be important to 
look at how teachers are involved, the time devoted to their involvement, and the areas of 
decision-making in which they are involved (e.g., budget, resources, staffing) (Louis, Marks, 
and Kruse, 1994). 

Thus far the classroom impact of school-based decision-making and teacher participation in 
decision-making is not clear (Louis, Marks, and Kruse, 1994). Also, actual restructuring in 
terms of management and governance has not been as prevalent as other forms of school 
reform efforts within schools that define themselves as 'Yestructuring" (Center on 
Organization and Restructuring of Schools, 1993). 

F. Linkages with Parents/Community 

As one final focus of change within a school, we include specific linkages and forms for 
interaction with the community and parents (e.g, U.S. Department of Education, 1994). . 
Examples are parent sessions that assist parents in understanding the school, linkages with 
community social services agencies and community youth organizations, and school-business 
partnerships. Through coordination with the community and parents, the school can 
increase consistency across the home/school approaches to working with student problems 
and increase resources available to children and their families. 

For example, a parent/community/school task force could work to develop common 
approaches for resolving problems and identify ways in which all could contribute to the 
solutions. Or, more challenging instruction for students can be supported through 
community linkages that offer experiences outside the classroom, e.g., opportunities to 
observe processes within a water treatment plant when the class has been researching water 
and its properties. Another option might be in the form of developing apprenticeships with 
local businesses where students can utilize skills they have developed in school. 

To assist students for whom health, economic or other needs are identified, the school can 
reach out to establish linkages with community social services, volunteer organizations, etc., , 
to ensure that students and their families can be linked with sources of help. Schools, in 
fact, are often used by hospitals, courts, and other social agencies as sources to help target 
students in need of special programs of assistance (Shann, 1990). Another type of linkage 
might be as simple as the principal or teachers visiting parents and members of the 
community, learning about their needs, and developing solutions that work to help both the 
community and the school (Lathrop, Vincent, and Zehler, 1993). 
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G. Student Instructional Experience 



The impact of any change effort cannot be known without having a very clear 
understanding of the instructional experience of students within that school. Simply stated, 
this requires a description of the instructional services received by students, a task likely to 
be more complex at the secondary than at the elementary level However, in both cases, 
understanding the instructional experience of students requires a student-level focus in 
describing services received, rather than a school-level focus. 

The distinction between a student-level and a school-level focus was made clear in the 
Descriptive Study of Services for LEP Students (Fleischman and Hopstock, 1993). In this 
study, schools were contacted to learn what types of instructional services were provided 
to limited English proficient students, with services defined as each unique combination of 
services that an individual student might receive. For limited English proficient students, 
this might include regular instruction in mathematics, bilingual instruction for language arts 
and social studies, English as a Second Language services, and regular music and physical 
education classes. However, researchers noted that in many schools, personnel found it very 
difficult to identify the combinations of services likely to be received by individual students; 
they were more prepared to provide information on the nature of different types of classes 
or services provided in the school. 

The definition of student instructional experience within an individual school most often will 
actually comprise several different possibilities. For example, there may be tracking of 
students into different sets of classes by ability; for language minority students, different 
levels of proficiency in English are likely to determine different sets of services (e.g., more 
proficient students will spend more time in regular classrooms for more subjects). A full 
description of the instructional experience of students should therefore include the structure 
of the program (classes, settings, groupings used), the curriculum received, nature of 
instruction (including language and instructional approach), and the student characteristics 
or groupings of students. 

1. Structure 

Particularly for language minority students who are not proficient in English, schools 
may provide a variety of service structures. For example, within small districts with 
only very few limited English proficient students, schools may share portions of a 
full-time English as a Second Language (ESL) teacher's time. The instructional 
experience for limited English proficient students might consist of regular classroom 
instruction for all subjects, with a pull-out class for ESL instruction, either on a fixed 
or flexible schedule. In other schools, students may stay within a self-contained 
classroom for bilingual instruction throughout the day, or they may participate in a 
two-way bilingual program. The structure of instructional experience would include 
the types of classes or instructional settings, the scheduling and amount of these, and 
the specific combinations of services received by individual students. Some teachers 
assign classroom "buddies" to assist limited English proficient students — an in-class 
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"structure". A "buddy" is often a peer of the same language group who helps the 
newer student become familiar with classroom rules and procedures; the ,T buddy" 
might also serve as a tutor or translator. Related to this informal model is a formal 
partnering program, Partners in Valued Youth (Robledo et al., 1990). In this 
program, middle school students serve as tutors for elementary grade students. 

The structure of a student's instructional day— how all of the different instructional 
settings are scheduled and coordinated — is an important indicator of a student's 
instructional experience. When students receive a variety of services in different 
settings, teachers are not always sure which students will be present with them in the 
classroom at a specified time (Hill & Kimbrough, 1981) and there is greater likelihood 
of fragmentation or disruption of instruction. In fact, current change efforts in 
compensatory education and in special education reflect a concern to decrease 
fragmentation of instruction (e.g., Ascher, 1988; Passow, 1989; Loucks-Horsley and 
Roody, 1990). 

Attending to the structure of a student's daily schedule therefore would be one 
means of addressing coherence. That is, in terms of the student's movement from 
one area of instruction to another, the change effort would involve examining the 
instructional periods scheduled for the student to ensure that students are guaranteed 
the opportunity to participate in the planned instruction in each class, and that there 
is a predictability for both teachers and students regarding the schedule of 
instruction. 

2. Curriculum 

The nature of the curriculum can be described in terms of content, level of materials, 
pace, depth versus breadth, integrated content versus separate subject areas, and 
student- and /or teacher-directed content. The relationship and comparability 
between the curriculum for limited English proficient students and the curriculum 
for other students would be an important component of describing the curriculum 
received by limited English proficient students. As Minicucci and Olsen (1992a,b) 
have reported, the curriculum received by limited English proficient students is often 
limited in range of subject areas and in level of coursework. 

Instructional changes designed to promote the development of higher-order cognitive 
skills are important means of presenting challenging content to students. For 
example, Warren and Rosebery (1990) worked with limited English proficient 
students who had previously only been learning science at a very low level of skills, 
as in rote learning of scientific terms. These students did not gain much 
understanding of science concepts or scientific modes of thinking. Through their 
work with these students, in conjunction with the content area and ESL teachers, they 
instituted a new approach to instruction that involved students learning to state 
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hypotheses, developing evidence, and interpreting data to answer questions using the 
scientific method. 

The content of instruction can be designed to relate to and build upon the students' 
current knowledge base by bringing elements of the different students 7 backgrounds 
and culture into the curriculum. Moll et al. (1990) have described one such approach, 
through utilization of the "funds of knowledge" available within the local community. 
Bringing the students' community into the classroom helps students to build 
understanding and appreciation of each others' cultures. Finally, efforts to address 
the coherence of a program would focus on curriculum across programs and across 
grades to ensure consistency across programs and articulation across grades. 

3. Nature of Instruction 

The instructional experience received by students would also be characterized in 
terms of the language used for instruction, opportunities for language use, and the 
instructional approach used. For language minority students, in particular, the 
description of instruction would include close attention to the nature of instructional 
language, including the use of the student's native language, but also focusing on 
opportunities for language use, and the nature of these opportunities (e.g., primarily 
routine, short, 1 expected" or formulaic responses versus opportunities to use language 
to negotiate content area meaning). Description of the use of language within 
instruction also should include the nature of the teacher's communication with the 
student, including whether the teacher uses the student's native language, adapts 
English language use to the student's level of ability, or uses standard English. Also, 
the teacher's use of repetition or paraphrasing or contextual support are aspects of 
instructional language use, as are the nature of question asked and the nature of 
discussions teachers hold with students. 

The nature of instructional tasks and of the interaction between teacher and student 
is an important component of the description of instructional approach. Identifying 
the roles and responsibilities of teachers and students within instructional tasks, 
indicating the degree to which instruction is student-centered or teacher-centered, for 
example, offers important perspectives on the teacher-students interaction. 

For example, cooperative learning approaches have been promoted as important 
forms of instructional tasks. Cooperative learning tasks provide opportunities for 
contribution by all students to the problem at hand through their sharing of 
information and skills. The implementation of a cooperative instructional approach 
must be carefully structured by the teacher to ensure that all students contribute and 
have the opportunity to share, and research would need to examine the extent to 
which all students do participate. Research is limited on limited English students' 
participation in cooperative learning approaches. However, Jacob (1994) reports that 
limited English proficient students do participate in cooperative groups and that they 
both help out other students as well as receive help themselves. Thus, in terms of 
instructional experience, implementation of . a cooperative learning approach is one 




example of an instructional approach which provides a means of increasing 
communication for limited English proficient students on meaningful topics with 
other students (including fluent English speakers). Effective implementation of this 
type of approach, however, would be based on teachers' skills and their beliefs that 
participation in cooperative groups supports the learning process for language 
minority students. 

Often, cooperative approaches are used when students are involved in student- 
directed problem-solving tasks, or "authentic" learning tasks (Newmann and 
Wehlage, 1993). In these types of approaches, students are active participants in their 
own learning and teachers serve as facilitators of that learning. 

4, Student Characteristics/Groupings 

Students may be assigned into homogeneous or heterogeneous groupings, or tracked 
according to academic level, or assigned into classes that comprise limited English 
proficient students versus primarily fluent English speakers. These different 
groupings constitute important aspects of individual student's instructional 
experience. The characteristics of other students with whom the student is instructed 
would be expected to interact with the various instructional approaches used and 
could play a role in the success of specific change efforts directed toward 
instructional approach and instructional activities. 

H. Summary: The Description of Change Efforts 

The description of change efforts has been presented within this chapter in terms of five 
areas of focus. These areas of focus in conjunction with the four goals of change presented 
in Chapter 5 together comprise our model for describing the content of change. Figure 6-2 
presents this model (introduced earlier), with examples of change efforts included. 

Using this model as a means of describing change, it is clear that change efforts directed 
toward a specific goal can vary considerably. However, if we try to define the efforts in 
terms cf the prii nary focus of the change activities (e.g., governance versus climate) directed 
toward a particular goal, we may be able to identify some general principles regarding 
which types of efforts are likely to be more effective. 

The description of the content of change in terms of the goals and areas of focus addressed 
by change activities, may make it possible to identify underlying general principles for 
successful change across a number of more specific instances of change efforts. Such a 
principle might state, for example, that activities directed toward one area of focus, perhaps 
climate /culture, should usually precede activities directed toward a second area, perhaps 
student instructional experience. Or, patterns of interaction might be identified among the 
areas of focus that can guide future change efforts. For example, if it is recognized that 
activities directed toward management/governance are best supported by specific school 
structures /characteristics, these can be taken into account in planning implementation. 
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Figure 6-2 

The Content of Change Efforts in Comprehensive School Programs 
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CHAPTER 7 

A FRAMEWORK FOR DESCRIBING THE PROCESS OF CHANGE 



A, Introduction / 

In this chapter we present our framework for describing the process of change in a school. 
Implicit in our framework is the idea that change is continuous and that any study of 
change will need to recognize that schools will change even in the absence of a clearly 
articulated model for change. When a model for change- does exist, however, there are 
certain stages and characteristics of the change process that can be systematically studied. 

Our framework for describing the process of change includes three major stages and six key 
characteristics of the change process. In the sections below, we briefly describe these stages 
and characteristics and discuss how they relate to each other. In the chapter which follows, 
we present some of the research findings on the process of change and provide more 
examples of key variables relating to these stages and characteristics. 



B. Stages of the Change Process 

A number of analysts have described different stages in the change process. They have 
referred to these stages as "steps," 'phases," or "stages," and have suggested different 
numbers of stages and different names for those stages. In general, however, three stages 
are most often identified: (1) initiation (mobilization, adoption); (2) implementation (initial 
use); and (3) continuation (institutionalization, incorporation, routinization). 

1. Initiation 

Initiation is the process resulting in a decision to implement a change. The initiation 
stage is typically defined as the period between when a decision is made to proceed 
with a change and when all or most of the planned change elements are introduced 
into one or more settings. The focus during the initiation stage is on mobilizing 
resources for the change, developing support for the change among the relevant 
participants, and planning specific activities or interventions to produce the planned 
change. 

2, Implementation 

Implementation is the experience of putting a leform or innovation in place with the 
expectation that other changes will occur. The implementation stage is typically 
defined as the period in which all or most of the planned change elements are in 
place in one or more settings. The focus during the implementation stage is on using 
the planned strategies to implement change, expanding the scope of the change effort, 
and examining the outcomes of the change. 
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3, Continuation 



Continuation refers to whether a change is institutionalized or disappears. The 
continuation stage is typically defined as the period after a decision is made to 
expand an innovation on a system-wide basis. The focus during the continuation 
stage is on institutionalizing the change through the training of new leaders, the 
development of standardized procedures, and the integration of the change into 
existing organizational structures. It should be noted that many change efforts do 
not become institutionalized, but elements of the change may be assimilated by 
individual teachers or groups. 

Obviously, this division of the change process into stages oversimplifies the complex 
developmental histories of most change efforts. The stages may overlap, and there 
may be multiple change efforts operating on different schedules. However, the 
differing events which analysts describe for these stages strongly suggest that a 
framework for describing the process of change include different research strategies 
for the different stages. 



C Characteristics of the Change Process 

In addition to describing the stages of the change process, a framework for describing 
change also has to describe a number of characteristics of the process. We have identified 
six characteristics which we believe are important to the framework: 

■ The context for change; 

■ The participants in change; 

■ . The scope of the change; 

■ The strategies for change; 

■ The organizational outcomes; and 
« The student outcomes. 

1. The Context for Change 

The organizational and social environment into which a change effort is directed is 
the context for change. It may include such factors as community attitudes toward 
specific educational approaches, existing organizational structures (e.g., academic 
departments), and grade levels served by the school. Typically the term "context" 
refers to variables which are not directly related to instruction and which are not 
thought to be part of a change effort. However, some approaches to restructuring 
have attempted to influence "context" variables either directly or indirectly. For 
example, efforts to create "schools-within-schools" have attempted to change the 
student experience of school size, which is normally considered to be a context 
variable. 
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2. The Participants in Change 



Who participates can vary significantly depending on the nature of the change effort. 
The change leaders can come from various levels, from district administrators to 
school administrators to teachers. The participants in change are also likely to differ 
across the different stages of the change process. The roles of district and school 
administrators, for example, are particularly important at the continuation stage. 

3. The Scope of the Change 

Scope refers to the breadth of the change effort at a particular point in the change 
process. The scope of change may be defined in terms of the number of participants 
involved, the number of innovations implemented, and/ or the numbers of goals and 
areas of focus addressed by the change. As with other characteristics of the change 
process, the scope of the change will likely vary across the stages of the change 
process as the numbers of participants, numbers of innovations, and numbers of 
goals and areas of focus addressed increases over the developmental history of the 
process. 

4. The Strategies for Change 

Strategies are based on the underlying assumptions concerning how changes are best 
implemented in organizations. For example, many change efforts are based on the 
assumption that changing the culture and attitudes within a school is a necessary first 
step in changing classroom practice. Other change efforts are based on the 
assumption that behavior changes often can and do precede changes in attitudes and 
beliefs. Depending on the strategy employed, the activities within specific stages of 
a change process will be very different. 

5. The Organizational Outcomes 

All of the various changes in the operation of a school system which result from a 
change effort are referred to as organizational outcomes. These may include changes 
in policies and procedures, changes in the interpersonal relationships within a school, 
Changes in linkages with parents, changes in student instructional experience, etc. 
These outcomes vary both in terms of whether they are direct or indirect 
consequences of the change model as planned and in terms of whether the outcomes 
relate specifically to student services or to other aspects of the school organization. 

6. The Student Outcomes 

Student outcomes are assumed to be the end products of all efforts. Student 
outcomes include those relating to student achievement of instructional objectives, 
thoSe we define as relating to student effort in achieving objectives (e.g., variables 
such as attendance, student engagement, goals for further education), outcomes 
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Figure 7-1 
The Process of Change 
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relating to student readiness to learn (i.e., good health and nutrition, other non- 
academic needs), and outcomes relating to native language proficiency and English 
language proficiency. Defining the appropriate student outcomes of a change effort 
would appear to be straightforward, but it is often one of the most difficult elements 
in defining a framework for a study of change. 

D. Describing the Process of Change 

The three stages and six characteristics of the change process are presented as a matrix in 
Figure 7-1. We use a matrix because we believe that the key characteristics of the change 
process vary across the different stages. We believe that this matrix provides a framework 
to guide the design of a study on school change. 

In the chapters which follow, we describe some of the research support and specific 
variables which underlie this framework. In Part IV of this paper, we summarize and 
integrate the frameworks to describe the content and the process of change. 
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CHAPTER 8 



THE STAGES OF THE CHANGE PROCESS 



A, Introduction 

As noted in Chapter 7, current research generally relies on a three-stage model to describe 
the flow of the change process. The three stages are initiation, implementation, and 
continuation. These stages are summarized below, focusing on variables that researchers 
have highlighted as important when studying the process of change. 



B» Initiation 

Initiation is the process resulting in a decision to proceed with implementation of a change. 
Fullan (1991) identifies the following factors associated with initiation: 

■ Existence and quality of innovations; 

■ Access to innovations; 

■ Advocacy from central administration; 

■ Teacher advocacy; 

■ External change agents; 

■ Community pressure, support, or apathy; 

■ New federal, state, or local policies and funds; and 

■ Problem-solving and bureaucratic orientations. 

Fullan suggests that the order of the factors is not important but different combinations are; 
for example, teacher advocacy combined with advocacy from central administration would 
be different from teacher advocacy combined with external change agents. Here, we group 
these factors and discuss related variables drawn from some key studies of change. 

1. Nature of Innovation 

The first two factors, the existence and quality of innovations and access to 
innovations bear on the nature of an innovation or potential change. According to 
the Rand Change Agent Study (Berman and McLaughlin, 1974), characteristics of 
innovative projects that may be particularly significant in deciding whether or not to 
initiate a change included: 

■ Perceived educational objectives, project techniques, and strategies; 

■ Perceived personal consequences; and 
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■ . Perceived institutional effects relating to: 

centrality (degree of displacement of central and routine behavior); 
consonance (fit between perceived educational goals and practices in 
the innovative project and perceived current goals and practices); and 
changes in resource allocation. 

Most of the research regarding access to innovations has assumed a lack of access 
and thus has focused more on the delivery side, for example, examining the 
characteristics of successful change agents or looking for ways to facilitate documents 
transfer. House (1976), however, suggested three measures that could serve as 
indicators for access to innovations or information about them: (1) the amount of 
interpersonal contact among staff within and across roles; (2) the presence of 
incentives /disincentives within the school district for innovation; and (3) the extent 
of access to outside personal contacts (e.g., professional organizations.) 

2. Advocacy for Change 

In the views of those who have studied education organizations, change does not just 
happen; someone either inside or outside the organization has to serve as the 
advocate of the change. In some cases advocacy couid be policy mandates from a 
higher level of government; in others it may take the form of a grassroots effort by 
teachers. What is assumed in educational change is that someone or something must 
give the organization a push to get it to change. Fullan (1991) listed five sources for 
that push: (1) advocacy from central administration; (2) teacher advocacy; (3) external 
change agents; (4) community pressure, support, or apathy; or (5) new federal, state, 
ox local policies and funds. 

One of the findings from the Change Agent Study was that the nature or content of 
an innovation was usually less important for its eventual incorporation or rejection 
than how it was initiated and implemented. A key factor in initiation was the role 
or position of the individual(s) who were its primary advocates. The Change Agent 
Study suggested that initiation of mandated changes would be resisted, at least 
passively, without local advocates. However, McLaughlin (1990) revised that 
assessment in later years because of the widescale acceptance and implementation of 
changes in federal education acts mandated by Congress, often without local support. 

3. Organizational Style 

Another aspect of initiation that is considered important is the style or culture of the 
organization, including the nature of its response to external policies and funds. 
Several characteristics of organizational style are associated with formation of 
effective advocacy groups, which in turn is related to successful initiation of change. 
Fullan (1991) describes this factor using a combination of terms with almost opposite 
connotations; he calls it "problem-solving and bureaucratic orientations." House 
(1976) lists three important characteristics of organizational style: (1) deformalization, 
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i.e., low emphasis on bureaucratic standards and procedures; (2) decentralization, i.e., 
power and decision-making is dispersed; and (3) unstructured role definitions, 
sometimes referred to as high boundary permeability. 

The Change Agent Study concluded its analysis of change by relating different patterns of 
behavior at each stage with behaviors at subsequent stages. For initiation, they found one 
variable to be most significant in explaining subsequent behaviors, i.e., whether the primary 
motivation for initiating a change was opportunism or problem-solving (Berman and 
McLaughlin, 1978b). When opportunistic motivations dominated, e.g., where the motivation 
was to take advantage of soft money or to enhance an individual's reputation, the 
innovation tended to founder during implementation and not be continued. Innovations 
that were adopted to solve problems were implemented more smoothly and were continued 
in some form. Other characteristics of the initiation of an innovation that were found to be 
significant for later stages included the substance and scope of the change and the choices 
made about implementation strategy (Berman and McLaughlin, 1975). 

C. Implementation 

Implementation is the experience of putting a reform or innovation in place with the 
expectation that change will occur. The complexity of the change process has resulted in 
researchers looking at implementation in two ways. The first involves listing key factors 
related to implementation success; the second lists the main themes of implementation 
success. Fullan (1991) notes the benefits of each method: "The former has the advantage 
of isolating and explaining specific roles; the latter is more likely to capture the dynamics 
of the change process" (p. 67). 

In line with the first approach, the key factors affecting implementation include the 
characteristics of change (e.g., needs, clarity of goals and means), local characteristics (e.g., 
social and organizational conditions), and external factors (e.g., mandates). Such factors are 
interactive and all simultaneously play a role during the implementation process of any 
educational change. 

As for the second way of looking at implementation, in recent years there have been a 
number of studies (Louis & Miles, 1990; Marsh, 1988; and Wilson & Corcoran, 1988) that 
describe the main themes in successful change in educational organizations. Their message 
is clear that a number of strong themes in combination make a difference in successful 
implementations. Fullan (1991) provides a list of themes adapted from Louis and Miles 
(1990) and adds one of his own: 

■ Vision building, including a vision of the school and a vision of 
the change process; 

■ Evolutionary planning or adapting change to fit a specific situation; 

■ Initiative- taking and empowerment; 
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■ Staff development and resource assistance; 

■ Monitoring /problem-coping; and 

■ Restructuring. 

According to Fullan, all six themes must be present during the implementation stage for 
substantial change to occur. The themes provide a dynamic and powerful image of the 
complexity of the implementation process for change. 

Findings from the Rand Change Agent study also stress the complex nature of the 
implementation process. Information collected through case studies of the implementation 
of innovative projects indicated that the projects changed through time as they were adapted 
to the unique aspects of the local situation and that the local situation adapted to fit the 
requirements of the new project. This process of what they termed mutual adaptation lies 
at the heart of successful implementation. Strategies they found to promote mutual 
adaptation included: 

■ Use of concrete, teacher-specific, and extended training; 

■ Provision of classroom assistance from project or district staff; 

■ Providing opportunities for teachers to observe similar projects; 

■ Holding regular project meetings focused on practical problems; 

■ Encouraging teacher participation in project decisions; 

■ Local materials development; and 

■ Principal participation in training. 

In contrast, unsuccessful implementations were characterized by nonimplementation (i.e., 
existing only "on paper") or cooptation, where something was implemented under the 
umbrella of the innovation, but it was not the planned innovation. Ineffective 
implementation strategies included: 

■ Use of outside consultants; 

■ Use of packaged management approaches; 

■ Preimplementation training on a one-time basis; 

■ Paying teachers for training; 

■ Emphasizing formal evaluation (particularly outcome evaluation); and 

■ Attempting to implement comprehensive projects (i.e., covering multiple 
schools, subjects, grades, etc.) (Berman and McLaughlin, 1978a). 

The Change Agent Study also highlighted the importance of looking at the complexity of 
the innovation, its "ambitiousness" (i.e., the level of "extra effort" needed), and the goals of 
the new project (Berman et al., 1977). Complexity, unUke comprehensiveness, was not 
necessarily related to ineffective implementation; in fact complexity could inspire teachers 
and others to work harder at the new project. In that way, complexity is tied to 
ambitiousness. One of the study's unexpected findings was the apparent willingness of 
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teachers to spend time and energy implementing a new project if they are convinced of its 
value. Project goals are important because they play a role in convincing teachers of the 
importance of the project (which has a direct effect on implementation), and they provide 
conceptual clarity, which is developed through mutual adaptation. 

As a stage in the change process, implementation leads to one or more decision points that 
will structure its continuation fate. Those decisions are influenced by the perceptions of 
participants and observers about the effects of the implementation not only on its targets 
(e.g., students) but also on other participants. These outcomes of the implementation stage 
are assessed at the school level and may involve factors such as: (1) the relative extent to 
which participants believed that project goals were achieved (i.e., perceived outcomes); 
(2) the type and extent of change in teacher and administrator behavior as perceived by the 
participants (i.e., changes in behavior); and (3) the extent to which implementation followed 
project design (i.e., fidelity of implementation) (Berman et al., 1975). 

D. Continuation 

Continuation refers to the institutionalization of change. Caution is noted in the meaning 
of continuation. For example, a school district may decide to discontinue a project but the 
teachers may have assimilated it; in effect, the very real continuation of the project may be 
officially overlooked. Fullan (1991) identified a number of factors that affect whether a 
change is institutionalized or disappears, including: 

■ Resources for staff development; 

■ Instructional staff support; 

■ Interest and support at the district level; and 

■ Principal support. 

Continuation of a project could range from assimilation without any formal local 
commitment to a conscious decision by a district to commit resources and extend the project 
to other schools. The decision to continue a project is typically based on whether it was 
perceived as successful, affordable, important to district priorities, and politically acceptable 
(Berman and McLaughlin, 1975). 

The decision to continue or discontinue an innovative project is not the final stage of the 
change process. For one thing, that decision generally takes place at the district level, but 
practices take place and become part of the routine within classrooms. For another, it is not 
clear that final continuation decisions are necessarily ever made, except in the cases where 
a project has been large enough to be widely watched or expensive enough that it is 
noticeable in budget hearings. Many innovations may simply be implemented on a small 
scale and never be formally resolved. Even when decisions are made, their intent may not 
be carried out. A project may be continued in name, for example, but be provided only a 
fraction of the resources from local funds that it needs once grant funds run out. As noted 
earlier, a project may even be discontinued at the district level but remain operational in one 



or several classrooms or schools where a group of teachers or principals remain convinced 
of its value. 

A term frequently used for project operation in some form following the continuation 
decision is institutionalization. Activities, practices, and even organizations can be 
considered as more or less institutionalized, and there are several dimensions along which 
institutionalization can be defined and measured. Some of these dimensions include: 

■ Duration (i.e., relative permanence of the activity within the organization); 

■ Formality (i.e., existence of written rules or titled roles associated with the 
activity); 

■ Awareness and acceptance of the activity by authorities outside the 
organization; and 

■ Development and maintenance of staff expertise to continue the activity 
(through hiring practices, commitment to training, and behavior monitoring). 



E. Issues Related to the Stages of Change 

According to Fullan (1991), the single most important idea resulting from this model is 
"change is a process, not an event" (p.49). The three-stage description of the process of 
educational change may involve numerous complications. First, many variables must be 
considered at each stage, and they may indicate different directions for the change. Second, 
the change process is not linear in that a decision made at one stage may unmake a decision 
made at a previous stage. Third, factors directly related to the characteristics of the desired 
change, including the initiator of the change and the sheer amount of time required for the 
change, are unspecified in the stages but will impact the change process. 

Fullan identifies four issues related to the process and implementation of change that cannot 
be easily categorized within one or another stage in the model: (1) active initiation and 
participation; (2) pressure and support; (3) changes in behavior and beliefs; and 4) problem 
of ownership of change. 

These issues, though not initially evident in the study of the change process, contribute 
significantly to the process. Each issue will be discussed briefly. 

1. Active Initiation and Participation 

Active initiation and participation is an issue that primarily must be considered at 
the initiation stage, particularly when large numbers of individuals are involved in 
the change. According to Fullan, active initiation and participation are key factors 
in the change process but sometimes, with large numbers of individuals, these actions 
are not implemented until the change process has begun. 
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2. Pressure and Support 



Pressure and support are necessary for successful change. During a successful 
change process, support and pressure become integrated. "Pressure without support 
leads to resistance and alienation; support without pressure leads to drift or waste 
of resources" (Fullan, p. 91). 

3. Behavior and Belief Change 

The relationship between behavior and belief change is ongoing and reciprocal. 
Changes in belief may precede or follow changes in behavior. When a change is 
implemented, the situation may regress before positive outcomes are experienced. 
Change in behavior and belief lead to understanding and the next issue of 
ownership. 

4. Ownership of Change. 

Ownership of change is a complex, progressive process. Ownership of change is not 
easily acquired. Individuals may be in favor of change, yet not be skiUed in the 
change or understand the change. Ownership results from a successful change 
process. It is a factor that must be developed from the beginning of the change 
process. 

Although change is a process, it is the individuals involved in this process who must 
be included in these issues of active initiation and participation, pressure and 
support, belief and behavioral changes, and ownership of the change process. 
Initiation, implementation, and continuation are dependent on these individuals. 
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CHAPTER 9 



THE CHARACTERISTICS OF THE CHANGE PROCESS 

In the framework described in Chapter 7, six characteristics of the change process were 
introduced. In this chapter, we describe some of the important variables related to those 
characteristics. 

A. The Context for Change 

The school is not an isolated unit. It exists within the context of a specific district and 
operates within district policy and management parameters as well as state-level policy and 
requirements. In particular, change efforts within a school will be influenced by the role of 
the district, by the change management approach taken by the district, and by the level of 
support provided by the district. In addition, the general community is a considerable input 
to and influence on the school, and planning for change efforts within a school must take 
general community politics and assumptions regarding schooling into account (Mirel, 1994). 

1. District Characteristics 

District characteristics, such as the enrollment size and the number and proportion 
of students who are language minority, will be important to note, since these will 
affect resources available in general, as well as have implications for the nature and 
amount of resources available for services to limited English proficient students. 

The district includes the district administrators, the school board, and other education 
decision-makers. It is important to examine their levels of support for change, their 
expectations, and their beliefs regarding what the nature of schools and schooling 
should be. These aspects of the district provide the general district climate for 
change; this may affect the local school community climate tor change, another 
organizational context factor. 

District-school relationship. In terms of the district-school relationship and the 
implementation of change efforts, two separate aspects of that relationship will be 
important. First, district-school sharing of management aiid decision-making will 
establish parameters for local decision-making regarding change efforts within the 
school. Top-down emphasis in decision-making will decrease the ability of the 
school to make decisions in adapting change efforts to its situation; bottom-up 
emphasis with management decisions based at the local school site will maximize the 
school's ability to define and manage the change effort. Some optimal blend of the 
two probably is best (Purkey and Smith, 1985). 

A second aspect of the school-district relationship will be the district administration's 
support for change efforts (Fuhrman et al., 1988; Mace-Matluck, 1987; Pechman and 
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King, 1993). For example, with regard to change involving bilingual programs in 
particular, Jones (1990) notes that for successful change to occur all district staff 
should feel ownership and support implementation of the programs, not only those 
identified as bilingual personnel 

The district provides support through encouragement and moral support for the 
change efforts and through the steps taken to enable the most effective 
implementation of the change. Examples of the latter are obtaining waivers or 
additional resources when needed, and providing access to budget figures, 
assessment data, and other information needed by the school staff for decision- 
making (O'Neil, 1994, based on a report by Wohlstetter). District support is likely 
be related to the nature of the school-district sharing of decision-making. In addition, 
Turnbull (1985) has noted that the nature of the support provided by a district to 
schools may be fragmented when the district environment is one that is too 
specialized or fragmented. Also, certain types of district support will be more 
effective early in the improvement process rather than later (Turnbull, 1985). 

Change efforts within the district. Another characteristic of the district would be 
the extent of change efforts within the district overall. Strategic planning for change 
within districts is discussed elsewhere in this report as an example of the nature of 
change efforts within the district. Rutherford and Huling-Austin (1984) have noted 
that the impetus for change within high schools in particular often comes initially 
from the district. Others, such as Fuhrman, Clune, and Elmore (1988) have noted 
that the district context for change is very influential; for example, districts are 
described as often proactive rather than reactive in responding to implementation 
standards (e.g., state standards). 

Here, however, we note that schools facing similar challenges in implementing 
change within a district are likely to provide support for one another. They can 
share experiences and learn from each other as they move through the various stages 
of implementation. 

2. Community Characteristics 

Outside of the school, the general community holds certain expectations for its 
schools, has assumptions about the appropriate goals and forms of schooling, and has 
beliefs about the effectiveness of the school for its students (Tyack and Tobin, 1994). 
These and various political viewpoints present in the community may work for or 
against change efforts within the school. Through communication and involvement 
of community representatives, change efforts can inform and try to obtain 
support — or at least acceptance — from various stakeholders within the community 
for steps the school is taking. The literature on reform efforts has presented findings 
documenting the role of the local community in supporting or denying the 
implementation of change (e.g., Mirel, 1994; Stiegelbauer, 1984). Therefore, the 
community is an important part of the school's organizational context when 
considering the description of change efforts. 
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The community can be described in terms of ethnic and language characteristics, SES, 
level of education, employment, education, etc. In addition, it can be defined in 
terms of existing involvement in the school, e.g., volunteering. The community can 
also be described in terms of community organizations and institutions that could 
provide important resources to the school. For example, focusing on language 
minority and limited English proficient students, community resources might include 
ethnic minority community organizations. Those organizations, which can provide 
support mechanisms for students and their families as well as translators to assist 
school staff, will play a role in determining the types of linkages that can be formed 
between the school and the community. The nature of the resources available may 
affect how successful and sustained those linkages become. 

3. State Policies 

Farther removed from the individual school, but still influential, are state policies and 
mandates. These can establish new parameters or new requirements for districts and 
schools which prompt changes within individual schools. One example, reported by 
Kauffman (1994), is that of Tuba City school, where the state policy and Tribal 
Council policy together led to a rethinking of the school's mission and to a new 
design for instructional programs. In order to respond to the state requirement that 
all students learn English plus another language and the Tribal Council requirement 
that there be Navajo language instruction, the school developed new programs 
including two-way English and Navajo bilingual classes and Navajo as a Second 
Language classes. 

B. Participants in the Change 

The "who" of change efforts is a separate and critical aspect of the definition of change. 
Included generally within the list of participants are the superintendent and other district 
administrators, the principal, the teachers (and teachers' union), the board of education, 
parents, community, and internal /external facilitators of change. Although most discussions 
do not focus on the students' role, the student is also a key participant (Gandara, 1994; 
Reyes and McCollum, 199?. McCollum and Walker, 1992). 

1. School District Administrators 

The policy and stance of the school district with regard to reform can play a very 
influential role in school reform (Fuhrman, Clune, and Elmore, 1988). Districts can 
be proactive in setting the mission and goals for reform; they may set up required 
plans for participation for schools (Felsen, Hopstock, and DiCerbo, 1994; Fuhrman 
et al., 1988). District administrators can also support school change by 
providing /seeking waivers that allow for implementation of reforms, that allow 
autonomy in decision-making to schools in areas of the reform, etc. Districts can also 
assist in getting state support for reforms by "making them [state policies] 
happen '(Fuhrman et al, 1988, p 254). 
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Superintendents can set in motion a sequence of strategic planning activities in order 
to begin a process of reflection and change within individual schools. Through 
various incentives or penalties the superintendents can push or encourage schools to 
enter the reform process (as described in the chapter here on district strategic 
planning). In addition, planning that incorporates language minority students and 
limited English proficient students in particular may depend on the nature of the 
district, its student population, and the district administrator's perspective. For 
example, even in a district with a relatively small proportion of limited English 
proficient students, the superintendent who recognizes the needs of these students 
can set in place the development of a plan and policy regarding their edr :ation. The 
particular format used may vary; it may be either in the form of a separate plan or 
through an emphasis on inclusion of limited English proficient students in all parts 
of the district's overall plan (Felsen, Hopstock, and DiCerbo, 1994). Either way, the 
district-level recognition and emphasis of the needs of these students will have 
implications for change efforts defined by schools within the district and for 
perspectives of school staff in serving all students. 

Much of the success of efforts provided by persons at the district level will be 
dependent on the nature of these persons' beliefs regarding the goals of the change 
efforts, commitment to facilitating and supporting the change efforts, and beliefs 
regarding the most effective way to provide this support. 

2. Teachers 

The characteristics of the teachers, such as their training, their beliefs regarding 
learning and regarding their roles and responsibilities, as well as their viewpoints 
regarding the change being implemented, will be important for how the reform effort 
is carried out Research on change in the past has often portrayed teachers as 
reluctant to change or resistant to change (Sarason, 1981). However, successful 
examples of change occur when teachers are brought into the process or work 
together to initiate the process (Richardson, 1990; Pechman and King, 1993). 

Teachers implement innovations within their own framework of beliefs, intentions, 
and theoretical premises which often will not coincide entirely with those of the 
research frameworks which underlie the innovations (Richardson, 1990). The 
implementation of change activities will then be different. It is critical to understand 
what local beliefs and local school cultures exist, both in the aggregate and in 
individual teachers, to understand how they related to the change effort overall and 
to implementation of innovations by individual teachers (Richardson, 1990; 
Goldenberg and Gallimore, 1991). Goldenberg and Gallimore (1991) provide 
examples from their research on how existing belief systems of teachers affected 
practices in the classroom and in expectations of parents. They showed further how 
eventual changes in those belief systems enabled the teachers to make changes in 
their activities in the classroom and with parents. 
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Individual teachers' views of their own role in the classroom and in the school, as in 
their feelings of efficacy in work with their students or in their role within the school, 
also appear to be related to implementation of innovations (Stein and Wang, 1988; 
Pechman and King, 1993). For example, the areao of decision-making in which 
teachers are given a voice within school-based management affect how the teacher 
views the value of that innovation and how the teacher views his/her own 
participation (Raudenbush, Rowan, and Cheong, 1990). 

Often change efforts begin with a core of teachers rather than all of the instructional 
staff in school. The specific group of teachers involved and the beliefs of these 
teachers will provide an important starting point for the description of a change 
effort. That is, the characteristics of this subgroup of teachers as distinct from the 
aggregate characteristics of teachers within the school will portray a more accurate 
picture of the input to and starting point of the change effort. For example, there 
have been several recent articles on the beliefs of mainstream teachers regarding 
language acquisition and on these teachers' perspectives regarding their role in the 
education of limited English proficient students (Faltis, 1993; Harklau, 1994; Penfield, 
1987; Faltis and Hudelson, 1994). The findings indicate that involving mainstream 
teachers in collaborative vision-building with bilingual and ESL teachers will require 
a change in beliefs about the process of language learning, about the types of 
instructional activities appropriate for language minority students, and a change in 
perspective regarding their own roles in instruction of language minority students. 

This will become a challenging task especially when the goal of a w r hole school 
program is to ensure that all teachers view the instruction of all students, including 
limited English proficient students., as part of their responsibility. Such efforts can 
be very successful, however. For example, Ortiz et al. (1991) has described the 
successful use of cross-disciplinary teams for addressing problems identified by 
teachers in working with students, especially language minority students. This team 
approach is a formally structured group which meets to address identified cases of 
students who are having difficulties in order to assess and develop appropriate 
instructional solutions for the students and to prevent their inappropriate referral to. 
special education. The team provides for a focused opportunity for sharing of 
information and perspectives across mainstream teachers and bilingual specialist 
teachers as well as other specialist staff members. 

Thus, a variety of research findings points to the importance of recognizing not only 
the overall, collective system of beliefs present within a school on the part of teachers, 
but also the belief systems of individual teachers and groups of teachers who are 
taking part in a change effort. 

3. Principals 

Principals are important participants in school change (e.g., Leithwood and 
Montgomery, 1982). The role of the principal and of principal leadership in change 
has been an important element within research on effective schools (Edmonds, 1979; 
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Brookover et al., 1978) and on change (Fullan, 1991). The literature on reform 
generally confirms the importance of the principal and expands on the various ways 
in which a principal's role can be described and the challenge of the position 
(Sarason, 1981). Some principals exercise "facilitative power" by serving as facilitators 
for their teachers, providing a form of leadership that encourages the role of teachers 
as professionals. Goldman, Dunlap, and Conley (1991) describe examples of 
successful change efforts in schools where the principal was able to stand back and 
allow teachers to make decisions. The principal assisted by providing resources and 
time for "creative" staffing to assist in fulfilling the staffs ideas. These principals 
were nonauthoritarian in their interactions with teachers and were important 
participants in the vision-building process, "removing barriers" as an ongoing activity 
to support teachers. 

There has been limited research related to principals as participants in change related 
specifically to limited English proficient students. Although many of the same 
findings would be expected to apply, the level of awareness of the principal 
regarding the needs of language minority and limited English proficient students, 
and effective instruction for these students would be expected to be important. This 
level of awareness would be critical if the principal is to serve as a leader and 
motivator with regard to whole school approaches that include equal focus on 
effective change for all students. 

4. Parents 

Most of the focus on parents and schools has been on describing parent involvement 
efforts and on attempts to increase parent involvement (Epstein, 1985, 1986). The 
types of activities in which parents are involved fit within parents' expectations 
regarding schooling and their role; most often these involve ensuring that parents 
provide support for homework, read to the students, and help out in non- 
instructional activities. 

With regard to change efforts specifically, parents are a key part of the community 
from which to obtain support for the change. When change efforts contrast with the 
parents' expectations regarding schools, then parents — and the community — will need 
to be convinced that ths changes are appropriate and their concerns will need to be 
addressed. In particular, parents will need to be informed and included in 
discussions if there is to be a "restructured" effort that contrasts with the general 
expectations and "definitions" of schooling that are assumed by parents and 
community. For example, using cooperative work groups and arranging desks in the 
classroom to support the cooperative work will puzzle those for whom straight rows 
of desks and individual student work define schooling. Where the parents and 
community see significant changes that do not mesh with their concepts of schooling, 
and where these sources of concern are not anticipated and addressed, the change 
effort is likely to run into considerable trouble (e.g., Mirel/1994). 
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In the case of language minority students, parent involvement efforts have often been 
designed to help inform parents of their role, to encourage them to take on a 
meaningful role in their children's schooling, and to assist them by helping them 
understand specifically what they can do to assist their child. In the case of language 
minority students, parents often do not understand what their role is in relation to 
the school due to their lack of familiarity with the system. For others, their cultural 
background leads them to assume a less participatory role in deference to the 
educators, whom they traditionally hold in great respect. 

Thus, the parents of language minority students may in some ways be more open to 
newer approaches to parent involvement, since they come as newcomers who are 
generally willing to learn about what their role can /should be. Change efforts such 
as those described by Moll et al. (1990) provide examples of parents coming into the 
classroom to provide their expertise to the class (e.g., a parent describing construction 
work on a house for a unit on buildings and architecture). Many other approaches 
have been tried out successfully to bring parents into the schooling process (U.S. 
Department of Education, 1994; Violand-Sanchez, Sutton, and Ware, 1991). The key 
is to understand the parent community and its needs and to design parent 
involvement approaches that meet these needs. 

For parents of language minority and limited English proficient children, language 
of communication with the school is an issue. Successful change efforts can 
incorporate language assistance and translation services at parent meetings in order 
to remove the language barrier as much as possible. Teachers often will need 
assistance in how to work with parents from different culture and language 
background*. This should include especially mainstream teachers, who frequently 
rely on the bilingual or ESL professional to handle all contacts with parents (Harklau, 
1994). 

5. Local Community 

In the discussion of change within a school, there has not been much written 
regarding the role of the local community, except in a reactive role. For example, 
while there has been a considerable amount of energy and activities devoted to 
school-business partnerships over the past decade, the specific role of the business 
and other community members and organizations as participants in a particular 
school's change effort has generally not been examined. However, in the context of 
strategic planning within districts, the community has been recognized as a key 
audience and stakeholder to be involved in discussions of change efforts from the 
beginning. 

When schools serve language minority populations, there are often within the 
community other types of organizations present with which the school may not 
typically interact. Often local ethnic groups will have developed a community group 
or association that serves as a central source of information and activities for 
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schools" established. These are schools which convene on Saturdays and which 
students from that language group attend in order to learn the language and culture 
of their group. For example, Japanese or Chinese students will attend a Saturday 
school in order to learn to read and write in Kanji and to learn about elements of 
their background and history that the parents and the community wish to preserve. 
These may be very rich sources of information, community linkages, and possible 
collaboration for the public schools. Thus far, given the focus on mainstream 
students, these types of community organizations have not been included in the 
discussion of change efforts. Similarly, language minority businesses and other 
organizations can be included in outreach and as sources of support for change 
efforts involving language minority students. 

6. Students 

The students involved in change efforts have not generally been included as 
participants; instead, their improved experience in instruction and increased 
achievement has been the focus of the change efforts. However, students play an 
important role. Student reactions to their restructured roles, to changes in their 
teachers' roles, to changes in the climate /culture of the school and to changes in the 
topics and materials used in instruction probably deserve more attention. 
Particularly at the upper grades (middle and secondary levels) student 
input/reactions to change efforts should be explored. 

Information can be available from language minority students as well as mainstream 
students regarding their instruction. For example, in case studies of services for 
limited English proficient students carried out as part of the Descriptive Study of 
Services for Limited English Proficient Students (Fleischman and Said, 1993), groups 
of students were interviewed regarding their instruction and their reactions. They 
were often quite frank in stating their views regarding the structure and scheduling 
of their instructional day, and the instruction they received. Student input as 
participants in change should be included in describing the process of change. 

7. Professional Organizations 

Teachers' unions often are asked to provide support of a planned change effort. 
Even when teachers' unions have given approvals of a project, problems may later 
arise that will weaken the change effort (e.g., Mirel, 1994; Felsen, Hopstock, and 
DiCerbo, 1994) as some of the specifics of what is needed to be implemented begin 
to take shape. Examples such as these show that the realities of the change effort 
in terms of contract terms, salaries, etc., where these might be affected need to be 
taken into consideration at the beginning of the change effort. Any concerns or 
sensitivities that are mentioned early in the process should be addressed immediately 
and directly. 

When the change effort involves language minority and limited English proficient 
students and specifically involves mainstream teachers views of their roles and 
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responsibilities related to these students, issues of burden may arise (as has been true 
for special education students). Although teachers already recognize that limited 
English proficient students are in their classes, change efforts that involve mainstream 
teachers' approaches to working with the students may need to be clarified with 
teachers and union representatives if any concerns arise. Staff development and 
ongoing support for all teachers would be an important concern in any case; 
information on assumptions and changes should be made clear to all teachers. 

8. Boards of Education 

Local boards of education are often key players in supporting change efforts, 
especially when there is a whole district effort involved. The findings from 
experience in implementation of change efforts indicates the importance of gaining 
support early on from Boards of Education, especially providing information and 
allowing input from the members of the Board to be taken into account in structuring 
the change effort. Clearly, in examining a change effort, the perspectives of the board 
of education and the implications of those perspectives for level of support for the 
district and for the school's efforts are important to understand and take into account. 

9. External Change Agents 

Some of the earlier research on change (e.g., Berman and McLaughlin, 1975a) 
suggested that external change agents were not sufficiently sensitive to local needs 
to provide helpful facilitation of change. However, as McLaughlin (1990) has noted 
in a reassessment of some of the earlier findings, external change agents can play an 
effective role in promoting change. Other researchers have also noted the role that 
persons outside of the school can play. For example, Jolly, Hord, and Vaughn (1990) 
report on a case study in which an external change agent assisted school staff in 
developing their vision of an ideal school, and then in specifying the tirst steps to be 
taken toward school improvement. Fullan (1991) also discusses the roles that can be 
played in change efforts by external agents. 

In some research on change and school improvement, researchers work with school 
staff or parent /community members in the conduct and study of the change efforts. 
For example, Shann (1990) carried out research on improvement efforts within middle 
schools through a collaborative effort involving university researchers and school and 
district staff. Delgado-Gaitan (1993) describes how she began as an 
observer /researcher and gradually took on the role of a participant in change in 
working with parents. 

In research on school change, the roles of the researchers, what information will be 
shared, and how it will be provided are important issues to consider. The degree of 
involvement of the researchers and the nature of feedback provided will involve the 
researcher to a greater or lesser extent as a researcher /change agent. His is likely 
to have implications for the design and findings to be obtained. 
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C. the Scope of Change 

Within the change process, it will be important to define the nature of the change effort's 
implementation in terms of its scope. Earlier, when we defined the content of change, it was 
pointed out that a balance must be achieved between the level of ambition of a change effort 
and the level of burden it brings to those involved. The reform efforts of the late 1980s and 
1990s began to focus on systemic change when it was recognized that implementation of 
numerous, discrete, small-scale innovations was not really tapping the potential tor 
meaningful change. However, there is at the same time a concern for not taking on too 
much at one time, so that the change effort becomes overwhelming. Thus, on the one hand, 
change efforts must be ambitious enough to be meaningful and to motivate and excite 
persons involved in the change effort. On the other hand, the change effort must not 
require too much too soon of those involved. Therefore, the question of the scope of change 
is a key part of the definition of a proposed change effort. 

Similarly, in describing the implementation of a proposed change effort within a school, the 
scope of implementation is an important question. A proposed change effort may involve 
a bundle of innovations but may be implemented initially on a much smaller scale. Ine 
scope of a change effort needs to be defined at different points in the mitiation, 
implementation, and continuation of the change effort. The scope of a change efforts 
implementation at any one point can be defined in at least four ways. 

1. Scope Defined in Terms of Numbers of Participants 

First, and most simply, the scope of the change effort can be defined in terms of the 
numbers of participants, or the proportion of possible participants in relation to those 
in the school. For example, a change effort may be designed to involve all teachers 
in a school. However, within the initial phase of implementation, the change effort 
may be designed to involve only a small subset of teachers, and then later expand 
to include all other teachers. In the experience of at least one group of researchers, 
working within the Coalition of Essential Schools, working with a smaller, separate 
group within a school as an initial step toward school-wide change does not work. 
When a separate segment within a school was used as a strategy for defining the 
initial scope of change, problems developed in terms of relationships with others in 
the school (McQuillan and Muncey, 1991, cited in Prestine and Bowen, 1993). Using 
a different but related categorization of scope, Rutherford and Huling-Austin (1984) 
described the scope of change in terms of four categories: individual, suburut, 
school-wide, and district-wide. 

2. Scope Defined in Terms of Extent of Use 

Scope of change has also been defined in terms of the extent of use of the change 
activities or innovation. For example, Tillema and Koster (1990) define extent of use 
as the degTee to which parts or elements of an innovation are used. Extent of use 



86 



9 

ERIC 



94 



could be defined in terms of frequency of use; it could also be defined in terms of the 
extent to which the innovation is fully implemented as opposed to implemented in 
part. 

3. Scope Defined as the Number of Innovations Implemented 

The implementation of a change effort can also be defined in terms of the number of 
separate innovations being implemented at one time. Often a change effort actually 
represents an "innovation bundle 1 ' (Loucks and Hall, 1981), and each innovation will 
require a process of initiating, implementing, and institutionalizing. Ideally, a set of 
innovations is always linked toward common, related goals. However, even if these 
are linked, it may not be possible to institute all of the specific changes at one time. 
Thus, the change process may be managed by a gradual process of implementation, 
in which selected specific changes are initiated first, and only after these are in place 
is implementation begun with others of the planned set of changes. 

For example, Hord (1984) describes "incrementalism" in implementation, building up 
a major change through a gradual process of implementing small changes. 
Depending on the nature of the change, however, this may not be possible or 
effective; instead, a more discontinuous, more ambitious change effort may be most 
effective. Also, motivation and support for the change effort may be stronger when 
there is a clear and ambitious shift to a new instructional approach or mode of 
operating within the school. 

4. Scope Defined in Terms of the Goals and Focuses Addressed 

An additional means of defining the scope of change activities implemented at 
different points in a change effort may be in terms of the model presented earlier in 
Chapter 4 in which change efforts were discussed in terms of four goals of 
comprehensive school programs and five areas of foe 3. An overall change effort 
within a school may be defined as addressing one or more goals by means of 
activities within one or more areas of focus. The activities implemented initially may 
be those that address only one of the goals; or, the activities initially may involve 
only one of the context categories and then expand to addressing other categories. 
Thus, use of the model for defining the change effort may also provide an additional 
way of identifying the scope of implementation at any one point in the change 
process. 

For example, a change effort may be designed to address the "challenge" goal by both 
introducing a new instructional approach that provides for student involvement in 
higher-order cognitive skills (student instructional experience), and by working with 
teachers to increase their level of beliefs and commitment regarding the need and 
potential for involving limited English proficient students in such academic tasks 
(climate/culture). There are at least two alternatives for the plan for implementation. 
One alternative would involve addressing only one of the components initially, i.e., 
either introducing the approach directly or working with teachers to build a change 
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in beliefs first (e.g., Stein and Wang, 1988; Conley, 1993a) before phasing in the 
actual use of the innovation. A second alternative would be to carry out both 
activities at one time. 



D. Strategies for Change 

The description of the change process within an individual school also will involve an 
understanding of the strategies for implementation that are underlying the specific change 
management approach. These strategies repiesent the beliefs about the most effective route 
to change on the part of the person or persons directing the change process. The description 
of these beliefs, and their implications for the strategies employed in management of the 
change process, may provide guidance for others in implementing it. The definition of such 
strategies implies that there is reflection on the change process and systematic planning for 
the implementation of the change effort. There are two types of strategies that might be 
defined: strategies for the design of implementation and strategies for support of change 
efforts. 

1. Strategies for the Design of Implementation 

As the literature has shown, effective change efforts most often involve 
implementation of more than a single, discrete change. Rather, it is more likely to 
be the case that there will be a related set of changes in the school organizational 
system (i.e., areas of focus addressed) and in the "schooling" received by individual 
students. Thus, change is relatively complex and implementation decisions are 
equally complex (e.g., Pechman and King, 1993). 

Correspondingly, the implementation of change can take a number of different forms. 
Those who are planning a change effort will need to determine the most effective 
means of initiating and implementing a change effort. One means of conceptualizing 
design options is provided by the model for the content of change, utilizing the four 
goals and five focuses of change. Steps to proceed could be defined in terms of the 
area of focus to be targeted initially, with a specific goal in mind. 

For example, a specific change effort may begin witu a focus on climate /culture 
factors, e.g., by motivating persons to recognize the need for change and to accept 
(or at least not resist) the fact that a change is about to be put into place. There are 
many discussions in the literature of steps taken to build the "readiness" (Conley, 
1993a) or the "will" (Miles and Louis, 1990) for change. Stein and Wang (1988) 
emphasize the importance of addressing the development of intrinsic interest and 
motivation on the part of teachers for carrying out the effective innovation (e.g., by 
providing data to demonstrate the need and by providing information on what can 
be done to reverse existing conditions). Using an alternative strategy, the initiation 
of a specific change effort would involve putting the change into practice with the 
expectation that beliefs and commitment to the change will follow. Thus, two 
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separate strategies could be defined: (1) address climate /culture first, instructional 
practice second; or (2) address instructional practice first, climate /culture second. 

It should be recognized, however, that the validity of an implementation strategy 
may vary for schools with different characteristics or within different settings. 
Specific strategies may need to be defined for particular schools types (e.g., a change 
within a school with a large bilingual program as opposed to change within a 
program in which limited English proficient students are most often within regular 
classrooms). However, over the course of a number of examples of change efforts, 
principles for the design of certain types of implementations in specific contexts may 
emerge. 

2. Strategies for Support of Implementation 

Equally important to the change effort are decisions regarding the nature of support 
provided for the implementation of a change effort. Staff development is one 
strategy for the support of changes within the classroom. Staff development should 
be designed to provide teachers with the ongoing support needed to effectively 
implement a new instructional approach or other innovation. 

Need for continuing support, Loucks-Horsley and Roody (1990) point out that there 
is need for continued training and support, as well as need for those implementing 
the change to recognize that change will take time. In addition, the needs of 
newcomers into a program for training and the needs of experienced teachers and 
staff for further growth both should be incorporated into any staff development plan. 

This same point is repeated by Stein and Wang (1988). The typical one-shot 
workshop approach cannot be effective in producing long-term change (Stein and 
Wang, 1988), since teachers will need continuing support from others as they attempt 
to carry it out in the classroom. Personal support is an important strategy for 
dissemination and is necessary for stable use of an innovation (Tillema and Koster, 
1990). 

Support for a new instructional approach might be through a mentor teacher making 
regular visits, through opportunities for teachers to discuss their experiences in 
implementation, or through other approaches. Support for teachers' participation in 
site-based management would include training in how to make effective use of 
committees, how to build consensus, develop budgets, etc. If such training does not 
occur, there is likelihood that the effort will become lost in power struggles, 
especially early on (O'Neil, 1994). 

Using goals and focuses of change to define strategies. The strategies for support 
of implementation will be shaped by the existing school characteristics /conditions, 
such as those defined within the five areas of focus of change. Characteristics of the 
school within one area of focus may have implications for success of a change 
directed toward another area of focus. For example, if the school's system for 
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management/ governance is a top-down system, using collaborative teacher groups 
might be a less effective means of supporting the innovative approach. Within a 
more "organic" system, in which there is already much teacher input and emphasis 
on professionalism, providing time for joint discussion and support focused on the 
progress of the change effort per se is likely to be more appropriate. 



E. Organizational Outcomes 

The term organizational outcomes refers to all changes in the operation of the school which 
result from the change effort, especially changes resulting from activities that address one 
or more of the five areas of focus (school structure/characteristics, climate /culture, 
management /governance, linkage with community/parents, student instructional 
experience). Researchers have indicated the need to develop a comprehensive set of 
measures to guide school reform and improvement (Shann, 1990; Oakes, 1989). Measures 
of school climate and other indicators of the nature of schooling within an individual 
institution provide important information that can be related to student outcomes. That is, 
measures of what is changed within a school as a result of change activities provide a means 
of tracking change and of linking changes accomplished within the school to later student 
outcomes. As an example, Lee and Smith (1994) have reported that a number of 
restructuring practices within secondary schools are associated with student achievement. 

Often, however, it is the case that the variables identified as important to a description of 
the school climate or of other variables that provide insight into the "internal life" of the 
school are also the most difficult to measure. Prestine and Bowen (1993) identify four 
benchmarks of school restructuring efforts within Essential Schools. They are the following: 
(1) substantial agreement within the school about the need for change; (2) observable change, 
i.e., visible change in the structure of the school; (3) all-school participation; and (4) systemic 
leadership (i.e., participants step into leadership roles and those in formal leadership roles 
also step aside to assume new relationships). They also note that there is no one identified 
model against which to measure the change processes since, within this model, each school 
must structure its own understanding of change. 

Given that the variables used to assess organizational outcomes are often also those that are 
difficult to measure, it will be important to develop multiple measures of key variables, and 
to include social as well as physical variables (such as size and resources) (Rutter, 1980, cited 
in Shann, 1990). Making a similar point, Jolly, Hord, and Vaughn (1990) state that schools 
are complex social institutions that require multiple assessment criteria. 

Teacher beliefs provide an example of variable that is more difficult to assess but which can 
have important implications for the services actually received by students. For example, a 
change effort may address changes in the beliefs of teachers regarding the level of 
instruction that can be provided to limited English proficient students as a preliminary step 
to involving teachers in providing challenging academic tasks to these students. A goal 
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would be to increase the level of challenge presented to students in their instruction. Direct 
organizational outcomes would be changes in teachers' beliefs regarding the potential for 
involving limited English proficient students in more challenging tasks. Indirect 
organizational outcomes would be actual use in the classroom of more materials and tasks 
that involve higher order skills, an outcome that would be more specifically related to 
student instructional services. 

As the preceding example indicates, there are two dimensions on which organizational 
outcomes vary: 

(1) Whether the outcomes are direct or indirect results of the change process; and 

(2) Whether or not the outcomes relate specifically to student services. 

1. Direct/Indirect Results of Change Efforts 

Any study of school change must necessarily recognize that change is a "messy" 
process and that the actual organizational changes which occur may or may not 
reflect those actually planned. When one part of the school changes, other parts also 
change~or actively resist change (Sizer, 1985); there are often "ripple effects" and 
many of these may help to build toward a broader change effort. A study of change 
should therefore focus on both direct and indirect organizational outcomes. For 
example, a change effort to increase communication among teachers of limited 
English proficient students may be effective in that regard. However, it may also 
lead to frustration with school administrators when common goals that are agreed 
upon by the teachers conflict with goals and objectives of the broader school system. 
Designing research measures which allow tracking of both direct and indirect 
outcomes thus should be an important objective of the study. 

2. Relation of Results to Instructional Services 

A second key dimension on which organizational outcomes will vary is whether 
those outcomes relate specifically to student instructional services. An example of 
an organizational outcome which does relate to student services is a measure of the 
challenge of the curriculum actually offered to limited English proficient students. 
A study could rate the complexity of curriculum materials being used in actual 
classrooms before and after a school change as a measure of change of student 
services. 

Other measures of the same change effort not specifically related to student services 
might include the number of teachers introduced to new curriculum materials and 
the amount of trainir £ offered relevant to those materials. A change study probably 
should include both types of measures, but the measures specifically related to 
student services are the most important Measures of changes in student services are 
often difficult to collect and thus are sometimes excluded from data collection, but 
any study of school change should clearly include them. 



91 

99 



It is also important to recognize that the organizational outcomes which should be measured 
will vary in priority across the three stages of the change effort (initiation, implementation, 
and continuation). During the initiation stage, for example, measures of changes not 
specifically related to student services (e.g., advocacy for change, the availability of external 
change agents) will be more important, since student outcomes cannot occur until there is 
first implementation of the proposed change. During the implementation stage, it is 
particularly important to collect measures of outcomes related to student services as well as 
measures of indirect organizational outcomes. During the continuation phase, there needs 
to be emphasis once again on changes not specifically related to student services (e.g., 
training of new staff, development of new leaders) in order to determine if the change effort 
will be long-lasting. 

The selection of specific organizational outcomes variables to be assessed will often be 
unique to an individual school, since they will be related to the specific goals and objectives 
that the school has selected for its improvement (Jolly, Hord, and Vaughn, 1990). In 
assessing change, it is also important to establish a reasonable schedule in implementation 
and in measurement of the changes, since change will take time and may occur slowly, m 
small increments (Pechman and King, 1993). 



F. Student Outcomes 

It is generally agreed that the ultimate goal of school change efforts is changes in student 
outcomes. However, beyond this general goal statement, it is often difficult to gain 
consensus concerning what student outcomes are desired and how progress toward those 
outcomes should be measured. As was true with regard to the organizational outcomes, it 
will be important to develop multiple measures of outcomes. 

The student outcomes that are selected should accurately reflect the educational outcomes 
that the district and school believe are most important and care most about (Lezotte and 
Bancroft, 1985). Student outcomes have most often been defined in terms of achievement 
test scores. However, the use of standardized assessment instruments for limited English 
proficient students has been a source of considerable debate, especially within the context 
of national standards and assessments (Rivera and LaCelle-Peterson, 1994; Zehler et aL, 
1994). More recently, student academic outcomes are also being examined through 
alternative assessments of academic outcomes (e.g., tasks that tap ability to think 
scientifically when the change effort has involved higher-order instruction in science, as in 
Warren and Rosebery, 1991; also, Navarette et al., 1990). These types of assessments offer 
greater possibilities in terms of assessing a broader range of student outcomes, through 
performance tasks, problem-solving tasks, and other means of demonstrating skills and 
understanding of concepts (e.g., "authentic assessment", Wiggins, 1989). 

For many researchers, the use of measures of cognitive outc nes only, and measurement 
of these by one type of test only (e.g., a standardized test, a test of reading) is too narrow 
a means of assessing change. For example, Shann (1990) found results in her research 
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indicating the importance of pro-social behaviors and other non-cog- utive student outcomes 
in relation to academic achievement. 

For the purposes of a research study on change, there will need to be agreed upon measures 
of student outcomes (cognitive and other) so that it is possible not only to gain an 
understanding of the effects of change efforts on the nature of the school and schooling 
received by students but also to link the change effort to changes in student outcomes. 

Based on the goals of programs that have been discussed thus far in this review, we suggest 
that the student outcomes to be observed and tracked are of the following four types: 

(1) Outcomes related to student achievement of instructional objectives (e.g., 
standardized tests, grades on classroom tests, "best efforts" in student portfolios, 
graduation); 

(2) Outcomes related to student effort in achieving objectives (e.g., school attendance, 
engagement in tasks, goals for future education, presence of products such as outlines 
and drafts in portfolios); 

(3) Outcomes related to student "readiness" (e.g., physical and emotional health, sound 
nutrition, well-rounded experience, etc.); and 

(4) Outcomes related to language proficiency (e.g., oral proficiency in English and the 
student's native language). 

1. Student Achievement Outcomes 

Student achievement outcomes are those that are most universally measured, often 
by standardized assessments. However, within a study of change the specific means 
of assessing academic outcomes may also differ across schools, as each school selects 
measures that most appropriately assess the skills and content emphasized in 
instruction. Thus, multiple measures of academic achievement would be important. 

2. Student Effort Outcomes 

In most models of school change, variables we define as related to student effort are 
necessary antecedents ' f student achievement, and thus measures of student effort 
are often included in studies of school change. By effort, we refer to a range of 
measures that relate to the student's approach to instruction and involvement in 
instruction, such as those that assess student engagement in academic tasKS, 
participation in cooperative work groups, improved attendance, increased positive 
attitudes toward schooling, and higher goals for future education. 
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3. Student Readiness Outcomes 



An ethic of caring or concern for the whole student is an important goal of schools, 
then measures of student outcome variables related to that goal are also important. 
These would be outcomes we define as student readiness outcomes, which would 
be assessed by variables such as extent to which students come to school in good 
health, receive support for problems at home, and receive care during out-of-school , 
hours. 

4. Student Language Proficiency Outcomes 

Within schools serving limited English proficient students, measures of language 
proficiency are key outcomes to be assessed. These would be measures of English 
language proficiency and, as appropriate within programs, of native language 
proficiency. These can be measures of oral language proficiency (speaking and 
comprehension) as well as literacy skills. 

We believe that any study of school change should include measures of all four types of 
student outcomes so that the linkage between organizational outcomes and student 
outcomes can be clearly established. A key question to be answered is whether changes that 
are implemented result in higher levels of student performance. This will be a difficult 
question to answer, since other factors, including other innovations, may all play a role in 
student performance at one time within a school. 

The selection of outcomes measures should be made so as to support school improvement 
efforts toward the school's selected goals (Torres, 1987). Student outcomes measures should 
also be carefully selected so that they are appropriate reflections of the schools' change 
efforts. Many studies err in their selection of achievement measures; often a measure is 
selected that is either insensitive to the specific innovations being implemented or too 
selectively keyed to a specific measure (Slavin, 1990). Research is needed that carefully 
selects and examines a range of student outcomes measures and that relates these to school 
change content and processes. 

It is also important to recognize that student outcomes are likely to differ at different stages 
of the change process. During the initiation stage, there may be great excitement about the 
change, and student outcomes may be positively affected by that excitement. On the other 
hand, the organizational changes may be not totally implemented, so changes in student 
outcomes may be delayed. During the implementation stage, there will be particular interest 
in student outcomes and in linking those with the change activities and organizational 
outcomes; thus, it is at this stage that student outcomes will need to be most clearly 
documented. At the continuation stage, it is likely that student outcomes will once again 
change as the effort expands beyond the original leaders and as adaptations of the content 
of the change are made. 
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PART IV 



A SUMMARY FRAMEWORK 
AND DESIGN ISSUES FOR A STUDY ON CHANGE 



103. 



CHAPTER 10 



APPROACH TO THE DESIGN OF RESEARCH ON SCHOOL CHANGE EFFORTS 



A. Introduction 

The preceding chapters have provided findings based on the research literature related to 
change. As this review has noted, the literature on school change has for the most part not 
focused on the issues of limited English proficient students or on the role of categorical 
programs such as those that serve limited English proficient students within reform efforts. 
Similarly, studies of services for limited English proficient students have typically not 
focused on comprehensive school programs. Clearly, however, given the goals that our 
nation has set for education for the year 2000 for all students, it is imperative that future 
research combine these two emphases. A study of institutional change affecting limited 
English proficient students should be an important contribution to this effort. Such a study 
will examine the nature and approaches to implementation of change with a recognition of 
the issues that will arise particularly within schools that serve limited English proficient 
students. 

As the research on change has demonstrated and as the recent Title VII legislation has 
indicated, there must be a broader perspective on the instruction of limited English 
proficient students. This perspective must look toward integration of services within a 
school and toward a more comprehensive concern with the quality of the schooling which 
these students receive. This argues for a research study that looks at change through the 
examination of the whole instructional experience of individual studenfs, and the activities 
and perspectives of all staff within a school. In this chapter, we present a summary 
framework for describing school change and discuss the issues that need to be addressed 
in the design of research on school change efforts affecting limited English proficient 
students. 

B. A Summary Framework for Describing School Change Efforts Involving Limited 
English Proficient Students 

In preceding chapters, we have focused on discussing the study of change efforts in terms 
of two main aspects: the content of change and the process of change. These together 
define a change effort. Examining process alone is not useful, since there must be a focus 
on the change activities themselves. However, examining the content of change without 
looking at the process of its implementation is similarly meaning less, since the process of 
implementation plays a key role in the final form and outcomes of the change. A change 
effort must therefore be viewed as comprised of both content and process. 

First, the content of the change effort should be defined in terms of the goal(s) addressed 
and the specific areas of focus through which the goal(s) will be addressed. In addition, the 
process of change must be defined in terms of the characteristics of the change process in 
each of the stages of change. Figure 10-1 presents a detailed representation of the change 
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effort in terms of both the content and process of change. We believe that this summary 
framework can provide the basis for the design of a comprehensive research study of school 
change efforts relating to limited English proficient students. 

C. Design Issues 

The summary framework for research on institutional change has been developed in order 
to guide the design of a study that will focus on change efforts within institutions that serve 
limited English proficient students. In developing the design for the research study, there 
are several issues to be addressed. These include issues related to the focus of the study, 
the sample to be used, the approach to definition of the change effort, the methodological 
approach, and the types of outcomes to be examined. These issues involved are discussed 
below. 

1. What should be the primary focus of the study on institutional change 
affecting limited English proficient students? 

These findings of this review on change efforts and the implementation of change 
indicate that there is much to be learned in implementing change within schools 
serving limited English proficient students. The study on institutional change should 
address the need for information on how change initiatives that focus on whole 
school integration of services can be effectively implemented when there are 
programs for limited English proficient students present. This recommendation is 
supported by the following findings: 

The limits of the existing research base. The review findings have indicated that 
there is very little work that integrates an examination of comprehensive school 
reform efforts and the specific efforts directed to limited English proficient students. 
Most of the literature on comprehensive school change has assumed a "universal" 
student and "universal" teacher. The larger portion of this literature involves 
collaboration of teachers, cross-classroom linkages, instructional approaches requiring 
student-student collaboration, teacher ?:acilitation of student efforts, and linkages with 
the community. Clearly evident in these efforts is a focus on building linkages and 
networks across what have been in the past traditionally isolated classrooms. 
However, most of the work has been carried out without a focus on the specific types 
of staff and classrooms involved. 

On the other hand, the research on instruction of limited English proficient students 
over the past two decades has developed a strong knowledge base of what 
constitutes effective classroom instruction for language minority students. Title VII 
programs, consistent with the focus in the research, have been defined in terms of the 
various classroom instructional approaches (i.e., two-way developmental programs, 
transitional programs, special alternative programs). However, despite research on 
characteristics of effective schools (Garcia, 1988, 1991; Carter and Chatfield, 1986), 
there has been very little work on instruction of language minority students that 
examines the development of linkages with other classrooms and services within the 
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school. The research on instruction of language minority students has not provided 
the information needed on "schooling" (Bidwell and Kasarda, 1980) or on whole 
school aspects of the program. More research is needed, for example, that looks 
closely at the instruction received by language minority students when they move 
into regular classes, either as part of "transitional" services or post-exit services, or 
that looks at coordination and collaboration among teachers across regular and 
special programs. 

The need for a research base. The research findings indicate that the newly 
authorized Title VII programs essentially will institute new whole school change 
efforts for which the existing research base is limited in terms of findings related to 
schools serving limited English proficient students. For example, we know a lot 
about collaboration and reflective practice among teachers in general but not about 
collaboration among bilingual classroom teachers and regular classroom teachers. 
We know that these teachers differ in beliefs about instruction and understanding 
about second language learning (e.g., Faltis, 1993; Harklau, 1994; Penfield, 1987) but 
not enough about how to implement change efforts to promote their effective 
collaboration. In addition, research has shown that there is too often segmentation 
of Title VII services from the .regular program of instruction, a fact that makes 
coordination across the regular and Title VII services more difficult. 

Thus, there is a need to have research findings that can be used to guide those who 
wish to begin the process of establishing an integrated, comprehensive system of 
schooling for their limited English proficient students. For this reason, the focus of 
a study on institutional change should be on the implementation of change efforts 
that are directed toward whole school change; that is, the efforts should be those that 
include not only effective classroom practices and student instructional experiences 
but that also include an integrated focus on linkages and relationships among 
classrooms, students, staff, and community. 

2. What types of schools and districts should be studied? 

Defining the scope of the study. To begin describing the scope of the study, we 
need to state just what specific group of districts and/or schools should be studied. 
Should these be only schools that are starting a new Title VII grant to support 
comprehensive school change? Should these be schools and districts that are 
nationally representative of all schools and districts, with only a smali portion of 
these Title VII schools /districts? In the case of the former, all schools included in the 
study would be Title VII grant recipients who were just beginning a five-year project 
to carry out a comprehensive change effort. In the case of the latter, the sample 
would not be limited to Title VII schools and would include change efforts as they 
randomly occurred in the sample. Each provides the basis for answering very 
different types of questions. 

A study of Title VII Comprehensive School Projects. One means of addressing the 
study of change would be to carry out a study that takes a "before" picture of schools 
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about to implement a Comprehensive School Program and tracks the change process 
for five years. The objective would be to determine the types of change efforts put 
into place and to identify any principles of implementing change that others might 
use in carrying out similar change efforts. In this approach, the study of 
Comprehensive School Programs would develop a research base from which others 
could learn. Thus, the program would help fulfill the role for the program as a 
"North Star" program as Title VII has been described by the OBEMLA Director, Dr. 
Eugene Garcia, 1994, i.e., to identify principles and practices that can inform other 
schools that are beginning similar change efforts. With the newly awarded grant 
projects as the focus of the study, there will be greater likelihood of opportunities to 
examine the implementation of change efforts focused on limited English proficient 
students and on the types of goals expected to be addressed by comprehensive school 
programs. 

A study of a nationally representative sample of schools. An alternative purpose 
of the study might be to describe what is occurring in schools now, i.e., to provide 
a descriptive study of schools serving limited English proficient at specified points 
in time over the five years. This study would presumably note and describe any 
change efforts that occur and their outcomes, based on the assumption that the new 
IASA legislation and general climate for reform nationally would promote such 
changes taking place. This type of study, however, would be less likely to focus 
directly on specific issues of change that would help to inform future change efforts 
in the field. 

3. What should be the relationship between the design of the study and the 
selection of grantees for the Title VII Comprehensive School Program? 

If it is decided that the recipients of Comprehensive School Program grants should 
be the focus of the research, there are possible interesting implications for the 
funding of the newly authorized programs. That is, the criteria for evaluation of 
proposed projects and the priorities for funding of projects could be defined in ways 
that will provide the best range of study sites and therefore best serve the goals of 
the proposed research study. 

For example, if the goal of the comprehensive school programs is to identify key 
approaches to implementation of change within a variety of contexts, the types of 
programs that are encouraged to apply may include those that represent not only 
large districts with large populations of limited English proficient students but also 
those districts that have smaller populations, with program structures that are 
correspondingly different. This might include schools where coordination is needed 
across bilingual classrooms and the regular classrooms into which students are 
gradually transitioned. It might also include districts where coordination involves 
an itinerant ESL /bilingual teacher and the regular classroom teachers who work with 
the students for the majority of the instructional day. 
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Also, using another approach, the review and selection of projects could be based on 
different priorities for the nature of the change efforts proposed and the 
characteristics of the local context in which they will be implemented. For example, 
change efforts that include a focus on the goals of challenge and coherence within 
specified types of school programs could be given priority. Thus, a purposive sample 
could be selected from among the projects funded as either the full sample for the 
study or as a significant portion of the sample to be defined. 

4. How should the change effort be defined? 

The definition of change efforts can be both specific and generic. This is true for 
descriptions both of the content of change and of the process of change. 

A specific definition of the content of a change effort would be in terms of the 
particular activities involved in the change effort within the school. At a generic 
level, the same change effort could be defined in terms of the general focus of the 
change effort used to address the identified goal. The generic level of definition 
would make it possible to look across different examples of change efforts and to 
discuss which tend to be more effective in promoting progress toward an identified 
goal, e.g., coherence in instruction. This would be, for example, distinctions among 
efforts that directly address the climate/culture in the school versus efforts that 
address student instructional experience directly versus efforts that address 
management/ governance. In other words, there may be general principles that can 
be identified as leading toward most effective change efforts. These principles may 
hold true within certain types of contexts (e.g., for a school within a large district 
having only a small population of limited English proficient students as opposed to 
a school within a large district with a large population of limited English proficient 
students). These types of close examinations of change efforts, carried out over time, 
could develop understandings of patterns that may inform practice (Cuban, 1990). 

With regard to the process of change, specific descriptions of the change process and 
its characteristics and stages could be used along with generic indicators such as the 
CBAM model (see Section D3 below). 

5. What should be the methodological approach? 

Given the complexity of the factors involved in the implementation of change efforts 
within any individual school situation, the design of the study should include 
methodologies that will allow a focus on the context, the content and process of 
change activities, and relationships /interactions among different variables and 
participants. Thus, case studies and ethnographic observations would be appropriate 
and important to include as at least one portion of the study to be conducted. 
However, if the study is limited to in-depth case studies alone, it will be difficult to 
draw conclusions that can be applied to other situations and thus less useful in terms 
of policy implications. Thus, the study design should be one that is two-pronged in 
focus, allowing for observations both across and within project sites. 



102 



Cross-site comparisons. First, the study should be designed to identify and address 
generic questions related to the implementation of comprehensive school change 
efforts. This could be accomplished by the selection of sites that allow for key 
comparisons in terms of the goals addressed, the strategies chosen for 
implementation of similar goals, different school characteristics (e.g., large school 
with a bilingual program versus one served by ESL instruction) or other context 
components. There should be across the sites some core sets of data obtained to 
allow for comparison in implementation and in outcomes. The study would 
therefore need to identify key issues and specify key variables that are consistently 
defined across the various sites. 

Within-site observation. Second, the study should focus on examining in greater 
detail the specific change efforts and specific implementation activities and outcomes 
and the relationships among these within each site. To the extent possible, the 
procedures for the case study or ethnographic efforts should include guidelines for 
observation of key indicators so that there is as much consistency as possible within 
and across sites. If the study approach within each site is similar, the observations 
and data obtained within individual sites may also be examined to draw conclusions 
regarding similarities and differences in patterns observed for different change efforts 
across sites. 

6. Should there be a longitudinal or cross-sectional design for the research? 

Ideally, perhaps, if a study is tracking a change effort within a school, it should 
longitudinally track the students who are affected by the changes. However, past 
experience in conducting research on language minority populations has 
demonstrated that the high mobility of the population makes it extremely difficult 
to carry out a longitudinal study. Nevertheless, two alternative approaches are 
possible: a comparison of different sets of students across a number of years and /or 
a comparison of the same set of students within a year. The former, i.e., the cross 
sectional approach, would allow for comparison across years of outcome data, 
including student outcomes. The latter approach would focus on outcomes found 
within a single year and would therefore be primarily focused on interim or 
organizational outcomes, although student outcomes could also be examined for 
those change efforts most directly related to instruction. 

Cross-sectional design. Cross-sectional comparisons make the assumption that the 
characteristics of students who enter a school are generally consistent from year to 
year. On the basis of this assumption, the effects of the change effort could be 
examined by means of student outcome tests and compared for trends. 

The cross-sectional design would most likely work as well for examining 
organizational outcomes. For example, if the change effort involves making 
individual students' instructional experiences coherent and not subject to disruption, 
the individual instructional experiences of students can be compared across years on 
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the individual instructional experiences of students can be compared across years on 
the assumption that the nature of the program and of student needs (and therefore 
combinations of services) remain equivalent from year to year. 

Student as own comparison. For certain change efforts, it may also be possible to 
make comparisons and look for outcomes within an individual year. As noted 
above, if the change effort was one closely related to instruction, then within-year 
changes in specific cognitive skills and tasks could be compared. Similarly, in terms 
of organizational outcomes, changes in the coherence of individual students' 
schedules from early in the year to later in the year would demonstrate change. 



D. Measurement Issues 

There are several additional issues to be addressed regarding approaches to the 
measurement of outcomes. These concern the levels at which variables are measured, when 
outcomes will be measured (i.e., the number of data collection points), and the appropriate 
measurement standards to be used for limited English proficient students. 

1. What are the appropriate levels of measurement? 

A major issue in the study of school change for limited English proficient students 
concerns the levels of measurement of important variables. A number of analysts 
have pointed out that if a variable is not measured at the level at which it is actually 
operating in a system, it is unlikely to be useful analytically. For example, studies 
often collect measures such as student /teacher ratios, per pupil expenditures, and 
socioeconomic level of the student body at the school level and then attempt to relate 
those measures to individual student achievement. In fact, there is often considerable 
variation within schools on those measures. Depending on grouping patterns across 
and within classes and the assignment of staff to those classrooms, the experiences 
of individual students within a given school can be very different. 

Similarly, a number of studies have made comparisons between "programs," even 
though experiences of individual students within specific programs have varied 
greatly. The effectiveness of a bilingual program would be expected to vary, for 
example, when there are differences in students' proficiencies in the native language 
and English, in their academic backgrounds in the content areas being covered, and 
when there are differences in their exposure to specific settings and times when the 
native language is used. 

Any study of school change thus will need to carefully consider the appropriate unit 
of measurement for each of the major measures selected. If a variable cannot be 
measured at the level at which it affects a student, the design of the study or the 
approach to measurement may need to be reexamined. 




2. What should be the frequency and timing of data collection? 

A second major issue complicating any study of school change concerns the 
frequency and timing of data collection. Two significant questions need to be 
addressed: 

(1) How often should data on outcome measures be collected? 

(2) What should be the timing and relationship of data collection on 
process variables and outcome variables?. 

Most studies of school change rely on simple two-wave (pre-post) designs. Data on 
outcome measures are collected before and after an educational intervention and 
results are compared. Analysts have identified a number of weaknesses of two-wave 
designs, however, and have suggested that more frequent data collection points on 
outcome measures are preferable. Multiple-wave designs allow the examination of 
trends, including growth curves and different patterns of growth among different 
subgroups of a population. 

There are problems with multiple measures of student outcome variables, however. 
If the measures involve testing, the costs in terms of time and money can be 
significant. Also, there may not be enough alternative versions of tests that measure 
the same construct to allow multiple testing. Multiple testing may also cause testing 
reactivity, such that the tests themselves are causing part of the change in results. 
Thus, for any study of school change there will need to be a careful examination of 
the advantages and disadvantages of using multiple measures of outcome variables 
across time. 

The study will also need to carefully consider the appropriate timing of data 
collection for process and outcome variables. Many analysts have pointed out that 
changes in outcome measures should not be expected immediately following certain 
educational interventions (training events, etc.). On the other hand, if outcome 
measures are not collected soon after the intervention or change, it may be very 
difficult to attribute changes in outcomes to the intervention. A solution to this issue 
is to study changes in student outcomes only after observed changes in 
teacher/classroom activities, but such changes may be difficult to observe and they 
may not occur on a schedule convenient to the study. It nriy also be appropriate to 
examine the sustained effects of an innovation well after the intervention has 
occurred. 

3. How should organizational outcomes be measured? 

There are two types of organizational outcomes measures that might be defined: 
generic measures and change-specific measures. 
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Generic measures of organizational outcomes. Generic measures of organizational 
outcomes are those that can be used across different change efforts, as a means of 
attempting to provide a common rubric for examining the progress of a change effort. 
An example of one generic approach for assessing organizational outcomes is of the 
Concerns-Based-Adoption Model (CBAM) (Hall, Wallace, and Dossett, 1973, cited in 
Loucks and Hall, 1981). The model is based on the kinds of concerns individuals 
experience in the course of implementation of an innovation and in the types of 
behaviors of users /nonusers of an innovation. Figure 10-2 presents the seven stages 
of concern and eight levels of use. Consistency and agreement on how to define and 
interpret specific instances of an individual's expressions of concern and type of use 
of an innovation would need to be obtained in order for these to be useful across 
sites. Also, these may be more appropriate as measures for one type of change effort, 
such as an innovative instructional approach than for other types, e.g., an increase 
in coherence in students' instructional scheduling. 

Another type of generic variable that has been defined is teacher self-efficacy (e.g., 
Ashton and Webb, 1986; Raudenbush, Rowan, and Cheong, 1990; Guskey, 1994). As 
Raudenbush et al. (1990) point out, teacher self-efficacy has been found by a number 
of researchers to be linked to effective teaching performance and ultimately to 
student outcomes. Raudenbush et al. (1990) following Bandura (1986) define self- 
efficacy as a person's judgment of his/her own capability to accomplish a given level 
of performance. One focus in the research has been on examining how school 
organizations can be designed to enhance teacher efficacy. For example, in research 
on self-efficacy of secondary teachers, Raudenbush et al. (1990) reported that self- 
efficacy was related to effective teaching and that it was sensitive to contexts in 
which teachers were working. Self-efficacy was related to teacher perceptions of 
students' engagement, the achievement level of the class, the teacher's level of 
preparation for teaching a particular class, and the degree of control over 
instructionally relevant working conditions. This variable may be particularly 
appropriate for certain types of change efforts, such as innovative instructional 
approaches and new governance structures. It may be particularly helpful in 
assessing change efforts in which regular classroom teachers are assisted in working 
with their limited English proficient, or recently exited students. These teachers 
would presumably begin with a low level of self efficacy with regard to instruction 
of their students who were not fluent in English. 

Specific measures of organizational outcomes. Greenwood, Mann, and McLaughlin 
(1975) denned implementation outcomes as "(a) the relative extent to which project 
goals are achieved; (b) the type and extent of change in teacher behavior; and (c) the 
extent to which the project as proposed compares with the project as implemented" 
(p. 4). This is one approach to assessment of organizational outcomes. Another 
approach is to examine what is present prior to initiation of a change effort, to define 
the nature of the changes expected, and to examine changes accomplished by 
comparison with what existed prior to the change. The specific types of measures 
to be used would be determined by the nature of the change effort, defined in terms 
of the goal of the change and the school context components addressed. Thus, for 
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STAGES OF CONCERN: 
TYPICAL EXPRESSIONS OF CONCERN ABOUT INNOVATION 





Stages of Concern 


expressions or v^oncern 


I 

M 


6 


Refocusing 


I have some better ideas about something that would work 
even better. 


P 
A 
C 
T 


5 


Collaboration 


I am concerned about relating what I am doing with what 
other instructors are doing. 




4 


Consequence 


How is my use affecting kids? 


T 
A 
S 
K 


3 


Management 


I seem to be spending all my time in getting materials 
ready. 


S 


2 


Personal 


How will using it affect me? 


E 
L 
F 


1 


Information 


I would like to know more about it. 


0 


Awareness 


I am not concerned about it (the innovation). 



LEVELS OF USE OF THE INNOVATION: 
TYPICAL BEHAVIORS 





Level of Use 


Behavioral Indices of Level 


VI 


Renewal 


The user is seeking more effective alternatives to the 
established use of the innovation. 


V 


Integration 


The user is making deliberate efforts to coordinate with 
others in using the innovation. 


IVB 


Refinement 


The user is making changes to increase outcomes. 


IVA 


Routine 


The user is making few or no changes and has an 
established pattern of use. 


III 


Mechanical Use 


The user is using the innovation in a poorly coordinated 
manner and is making user-oriented changes. 


II 


Preparation 


The user is preparing to use the innovation. 


I 


Orientation 


The user is seeking out information about the innovation. 


0 


Nonuse 


No action is being taken with respect to the innovation. 



From Loucks and Hall, Investigating Program Implementation: A Field Perspective, p. 5. 



107 



example, if the school were addressing the goal of increased coherence in the 
instructional experience of students and taking steps to ensure a smooth flow of 
instruction for each student, the pre-change measure might be a variable that rates 
the coherence of each student's instructional day in terms of lack of disruption of 
instruction. At different points in the process of the change effort, these ratings of 
individual students' instructional experience would be obtained again to examine 
whether there was any of the expected change toward less disruption of instruction. 
The ultimate assumption in examining and tracking organizational outcomes, of 
course, is that achievement of the planned changes will lead to improvement in 
performance and achievement outcomes which the school has identified for its 
students. 

4. What are appropriate standards/outcomes for limited English proficient 
students? 

An extremely difficult and controversial issue is related to the question of the 
appropriate types of outcomes measures and standards appropriate for limited 
English proficient students. As noted earlier in Chapter 9, there has been 
considerable disagreement in the field about what measures should be used to assess 
schooling success for limited English proficient students, and when and if those 
measures should be applied for new limited English proficient entrants into U.S. 
schools. With regard to standardized testing in particular, the disagreement has 
centered around three questions: 

(1) Should native language versions of tests of content area knowledge be used? 

(2) Should limited English proficient students be exempted from testing in 
English until they reach specific levels of English language proficiency? 

(3) Should tests for limited English proficient students (or all students) be 
redesigned so that they include items more relevant to students' cultural 
backgrounds? 

Those questions are embedded in debates about goals of educational programs for 
limited English proficient students. On one side of the debate are those who argue 
that primary goals should be rapid learning of English and acquiring a "core of 
knowledge" defined by the majority culture. Advocates of that position emphasize 
the use of traditional tests in English for all students. On the other side of the debate 
are those who argue that the goals for limited English proficient students should 
include fluent bilingualism and the acquisition of knowledge that is not culture 
bound but can be applied in multicultural situations. Persons on this side of the 
debate emphasize testing in the native language, testing of content knowledge in 
English only after English proficiency is strong, and testing that recognizes the 
cultural knowledge of limited English proficient students. Although these arguments 
have been made with regard to standardized achievement tests in particular, they 




have also been raised in conjunction with alternative assessments, such as 
performance assessments. 

Any study of school -wide efforts for limited English proficient students will need to 
confront this issue and to select outcome measures that are both broadly accepted by, 
and fair to, those students. Finding a balanced approach to this issue will be very 
difficult, and even a balanced approach is likely to be criticized by advocates on both 
sides of the issue. 

Within schools that are implementing change and that are included in the sample for 
the study, there will be student outcome measures in place. These will have been 
determined by the school and /or the district, and influenced by district, state, and 
national standards, policies, and guidelines, and designed with already determined 
assessment procedures and local outcomes measures and goals in mind. These 
outcomes me isures should be available within an individual site, and comparison 
data for previous cohorts of limited English proficient students may be available. 
Outcome data for fluent and monolingual English speakers may also be available. 
Thus, local outcomes measures may be one source of data to use. However, it may 
also be useful and appropriate to include in the design assessment measures that are 
consistent across different sites, particularly if there are similar sites with similar 
change efforts being employed. 

E. Analytic Issues 

A major issue complicating many major studies of services for limited English proficient 
students is the complexity of the educational environment and the lack of control over most 
of the key variables under study. The result has been that large volumes of data have been 
collected but that analysts have been unable to make clear causal inferences. It is therefore 
important to attempt to define expected relationships among key variables and to examine 
data that will bear on these relationships. To deal with this issue, it is important to 
recognize that the relationships among variables influencing services to limited English 
proficient students are extremely complex. Therefore, approaches that attempt to explain 
the effects of all of those services on all populations, even if they use complex statistical 
modeling, are unlikely to provide clear answers. What is needed are more narrowly focused 
analyses in which the effects of different narrowly defined service patterns (e.g., integrating 
ESL services as a supplement to, rather than as a replacement for, regular English classes) 
are examined within narrowly defined subgroups (e.g., students with very limited English 
skills, with strong proficiency in their native language, and /or with no previous English 
language instruction). Below are some of the analytic issues involved in examining the 
effects of change efforts. 

1. What should be the focus of analyses? 

Analyses can be focused on examination of change efforts within specific types of 
schools or with specific types of participants (students, staff, community). Care 
should be taken to recognize that the particular participants in a program or change 
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effort represent a particular case and that what is true for one group may not be true 
for another. For example, educational services are in most cases directed to those 
students who can make best use of them, and thus, 'services in the native language 
are most often offered to limited English proficient students whose proficiency in 
English is weakest. As a consequence, students who learn English most quickly exit 
such services, while those who learn English more slowly remain. Thus, there is 
typically a negative correlation between the amount of native language use in the 
classroom (particularly beyond the first or second year of instruction) and 
performance on tests in English. The negative correlation is mostly or completely an 
artifact of the way students are assigned, so it is difficult to draw strong inferences 
about the effectiveness of native language services. A similar problem confronts 
research involving compensatory education services. 

Similarly, change efforts for students may have different effects depending on the 
characteristics of students. For example, if a change effort is directed toward 
cooperation and collaboration between a bilingual education teacher and a regular 
teacher, it may be easier to establish this cooperation when students are more 
proficient in English and therefore closer to the expectations of the regular classroom 
teacher (and therefore more within the ran-^e of that teacher's "responsibility") than 
when the student is at an extremely low level of proficiency. Thus, examination of 
specific change efforts implemented with different types of participants (students, 
teachers) would be one focus of the analyses to be employed. 

2. How should the analyses examine alternative implementation strategies? 

A second analytic issue concerns the examination of specific implementation 
strategies. The research clearly shows that the implementation process is a critical 
aspect of change efforts. At the same time, the descriptions of various change efforts 
in the literature identify a variety of implementation strategies (e.g., in terms of scope 
at different stages of the process, in terms of a specific area of focus addressed, in 
terms of support strategies). Thus, focused analyses of specific types of strategies 
employed in the implementation of change efforts may lead to identification of useful 
principles for implementation of change. For example, if projects directed toward 
coherence of instruction consistently show that addressing instructional practices as 
a first step is better than either addressing beliefs first or addressing both of these 
context components simultaneously, others will be able to use this principle in 
designing their own implementation strategies for similar change efforts. If studies 
focus on selected relationships such as these, a knowledge base regarding principles 
for implementation strategies for sets of change activities may be gradually 
developed. 

F. The Role of Researchers and Participants in the Study 

A final research design issue concerns the role of outside researchers and participants in 
ongoing change efforts. The literature on change has emphasized the importance of local 
contextual factors and local knowledge and assumptions on the part of the those involved 
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in the change effort. Also, the participants in a change effort have been identified as a key 
aspect of a definition of change. Should local participants also become involved as 
researchers? On the one hand, those most closely associated with the local school 
understand best the local context and factors that will be important to the change effort. 
They will also have the day-to-day experience of carrying out the change effort and can 
report in detail on its progress. However, those who are directly involved in the change 
effort also will have the most difficulty in reporting objectively. There may be a need to use 
some combination of approaches both to take advantage of the insights of those involved 
and also to obtain data collection and observation at selected points by outside researchers. 
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I. INTRODUCTION 



A focus group on the design of a study of school change affecting limited English proficient 
(LEP) students was convened by the Special Issues Analysis Center (SIAC) on January 11-12, 
1995 at Development Associates, Inc., Arlington, Virginia. The meeting was held to discuss 
options for a Benchmark Study of school change which is being sponsored by the Office of 
Bilingual Education and Minority Languages Affairs (OBEMLA) of the U.S. Department of 
Education. The study is intended to provide policy-related feedback on Title VII of the 
Improving America's Schools Act (IASA) (especially the Comprehensive School Grant 
Program) and to inform local educators concerning effective approaches to serving LEP 
students. The purpose of the meeting, as denned by Dr. Eugene Garcia, Director of 
OBEMLA, was not to design the study, but to frame the questions, domains, parameters, 
and issues which the researchers should consider in designing the study. 

Nine experts on research on school change and the evaluation of programs for LEP students 
participated in the focus group. The participants included university researchers, program 
evaluators, and persons with responsibility for program development and policy direction. 
Over the course of the two-day meeting, the participants addressed questions in five topic 
areas: (1) research questions for a study of school change; (2) models for studying the 
content of school change efforts; (3) models for studying the process of school change efforts; 
(4) methodological approaches to studying school change; and (5) methodological issues in 
studying school change. Each broad area was broken down into more specific questions 
which were addressed in group discussions. The participants also provided written 
recommendations in each of the five areas. 

The focus group report consists of two volumes. This volume, Volume 1, presents the 
findings of the focus group meeting. The report describes the oral and written comments 
and recommendations of participants concerning the design of the Benchmark Study. The 
goal of the meeting was not consensus among participants, so the report describes areas of 
agreement and disagreement. In addition to the findings, Volume 1 also contains a list of 
focus group members (Appendix A), the schedule and agenda for the meeting (Appendix 
B), and the individual written recommendations of the nine focus group members 
(Appendix C). Volume 2 contains the transcript of the two-day meeting. 
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II. ABSTRACT 



The focus group discussion was organized around a series of questions formulated by 
OBEMLA. The questions and summary answers to them are presented below. 

Given the government's stated purpose for the intended research, what framework(s) 
would be most appropriate for the design of a study examining refonn and change within 
schools serving LEP students? That is, what recommendations are there regarding 
theoretical assumptions, regarding the definition of effective change efforts, and 
regarding key study questions and variables to be used in the study? 

The study as described by OBEMLA has both evaluation and research purposes. The first 
purpose is to evaluate the new Comprehensive School Program as funded under Title VII 
of the Improving America's Schools Act (IASA) to indicate future policy directions. The 
second purpose is a research study to examine effective approaches to comprehensive school 
efforts for LEP students in order to inform practitioners. The focus group members differed 
in the relative emphases which they would give to the evaluation and research purposes of 
the study. 

Some focus group members proposed designing a research framework that would explore 
effective approaches to comprehensive school change within an evaluation of the Title VII 
Comprehensive School Program. A second group, although aware of the need for an 
evaluation, stressed the overriding value of the research purpose. These focus group 
members, speaking from the viewpoint of practitioners in the field, argued that a study 
focused squarely on comprehensive change efforts would be more likely to add to the body 
of research knowledge about how to bring about such change in schools. The focus group 
members expressed concerns about the ability of one study to address both evaluation and 
research purposes, especially with limited resources. 

One focus group member defined a question which related the evaluation and research 
purposes. "What are the most effective ways to bring money into situations which will 
leverage the kinds of changes which we, as a collective group of educators, think are 
representative of best practices?" Focus group members did not believe that strong causal 
conclusions could be drawn between Title VII funding and outcomes, but they did believe 
that the study could describe the "footprint' 1 or "value added" by Title VII funding. Focus 
group members indicated that the focus of the study should be both on the content and on 
the process of change. 

Focus group members recommended that the study include both quantitative (surveys) and 
qualitative (case studies) strategies. They indicated that the exact mix of the two could not 
be determined without definition of the study questions and the amount of resources 
available for the study. Focus group members suggested that there should be a "core" study 
which should remain in place throughout the entire study process, but that there should also 
be flexibility to address new questions /issues as they emerge. 
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Most focus group members recommended that samples of both Title VII and non-Title VII 
schools should be included in the study. On the other hand, some focus group members 
disagreed on whether a comparison group of non-Title VII schools was needed in order to 
draw conclusions about Title VII schools. Without knowing the specific study questions to 
be addressed nor the amount of resources available, focus group members could not define 
how many schools should be studied. 

Given the government's stated purpose for the research, what specific measures and 
methodologies would yield valuable and reliable information about the nature and impact 
of reform efforts within schools, especially those serving LEP students? 

Focus group members suggested that it is difficult to define standards related to the content 
of change because such standards reflect the values of a particular group. Focus group 
members generally agreed, however, that school change could be judged based on the extent 
to which there was: (1) challenging or ambitious academic content for students; (2) 
coherence in the instruction provided; and (3) a climate of collaboration among staff. 

Focus group members used different categorization systems to define the range of key 
content variables to be studied. The various systems used, however, generally included 
variables in eight categories. The categories were: (1) national, state, district and local 
context; (2) school structure and characteristics; (3) school climate and culture; (4) teacher 
characteristics; (5) management or governance structure within the school; (6) school 
linkages with parents and/or community; (7) student instructional experience; and (8) 
student characteristics. A number of relationships among key variables were emphasized 
by specific focus group members, but there were few common themes across focus group 
members. 

Focus group members disagreed on the extent to which the study should focus on student 
outcomes. Most focus group members did not believe that the cost of developing and 
validating new student achievement tests was justified for this study. Focus group members 
raised concerns about realistically assessing student outcomes in relation to the changes 
brought about through the Title VII Comprehensive School Program given the nature of the 
change process and the relatively short time period of the proposed study. In examining 
outcome data for LEP students, focus group members stressed that serious efforts need to 
be made to separate the variance due to content area knowledge from the variance due to 
language proficiency. 

Given the government's stated purposes for the research, what specific measures and 
methodologies would yield valuable and reliable information about the key components 
in the process of implementation of change, including factors that promote or impede 
change? 

In general, focus group members indicated that they did not believe that the study should 
apply standards for the process of change. Focus group members suggested a number of 
categories of process variables which they thought should be examined by the study. These 
categories were: (1) initiation and development of a shared set of change goals; (2) 

3 
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development of mechanisms to support change such as professional development and 
resource assistance, e.g., time, technical assistance, training, funds; (3) planning for change, 
including setting strategies; (4) monitoring /problem solving as the change is implemented; 
(5) adjusting /revising change goals to the situation; (6) evaluating the success of the change; 
and (7) institutionalizing the change. 

What approaches would you recommend for collecting data over time that focusses on 
teaching/learning environments; on teacher/learner relationships? What are the design 
requirements of such approaches? 

Focus group members suggested that case study assessments of classroom activities include 
classroom observation, teacher interviews before and after class, examination of curricular 
materials used, and review of actual student work during the class session. Focus group 
members generally agreed that classroom observations should occur a number of times over 
the school year in order to capture change in the learning environment and teacher-student 
relationships. Focus group members also suggested that standardized protocols should be 
used to help observers capture and record activities in the classroom, and so that cross-site 
analyses could be performed. Focus group members stressed the importance of using 
classroom observers who are knowledgeable about the content area being taught and who 
are proficient in the language of instruction. 

Focus group members urged that pre-class teacher interviews include discussion of the goals . 
and objectives of the class period and of the strategies to be employed. Post-class interviews 
would include discussion of goal achievement and reasons why specific classroom activities 
took place. Focus group members urged that student work products from the classroom 
should be examined in order to judge if that work is of "rigorous" quality. Participants 
suggested that small group interviews with students should also be used to assess their 
perspectives on instruction, student-teacher relationships, their general views of the school, 
and their views on themselves and their schoolmates. Students should also be observed in 
non-classroom settings such as libraries, hallways, cafeterias, etc. 

For the quantitative component of the study, focus group members suggested the use of 
teache: questionnaires and teacher logs to study classroom activities. Participants also 
suggested the use of teacher questionnaires to examine such factors as teacher collaboration 
and school climate. 

What is the recommended role of investigators in change research regarding one's level 
of involvement, scope of contribution, objectivity and professional ethics? 

In order to maintain a positive working relationships, focus group members believe that 
investigators need to share some of their observations with school staff. One member stated 
that interactions between researchers and their subjects typically result in exchanges of 
information, such as providing information on references to check, contacts to communicate 
with, or informal reflections on instructional and teaching practices. On the other hand, 
focus group members do not believe that investigators should be active agents of school 
change. 



III. FINDINGS 



The purpose of the focus group discussion was to provide OBEMLA with a set of ideas and 
options for the design of a study to measure institutional change affecting the education of 
LEP students. First, we present the panelists' thoughts on study purposes and study 
questions, and the general approach to the study. Next, we present a synthesis of the views 
of focus group members concerning appropriate models for studying school change. This 
discussion is centered around two main issues: (1) the variables and standards for assessing 
the content of educational change efforts; and (2) the variables and standards for assessing 
the process of educational change. Finally, we present a summary of the panelists' 
recommendations for specific methodological strategies to be used for the study. 

A. Study Purposes and Study Questions 
1. Purposes of the Study 

The study as described by OBEMLA has both evaluation and research purposes. The first purpose 
is to evaluate the new Comprehensive School Program as funded under Title VII of the 
Improving America's Schools Act (IASA) to indicate future policy directions. The second 
purpose is a research study to examine effective approaches to comprehensive school efforts 
for LEP students in order to inform practitioners. 

The focus group members differed in the relative emphases which they would give to the evaluation 
and research purposes of the study. Some focus group members proposed designing a research 
framework that would explore effective approaches to comprehensive school change within 
an evaluation of the Title VII Comprehensive School Program. In this way, the study would 
comply with the legislative mandate for an evaluation while, at the same time, answering 
questions of importance to educators. 

A second group, although aware of the need for an evaluation, stressed the overriding value 
of the research purposes. These focus group members, speaking from the viewpoint of 
practitioners in the field, argued that a study focused squarely on comprehensive change 
efforts would be more likely to add to the body of research knowledge about how to bring 
about such change in schools. As one focus group member said, practitioners 
— administrators and teachers — need to know "how one makes the most effective use of 
scarce resources to provide the conditions for effective practices and [to] improve 
instruction." 

The focus group members expressed concerns about the ability of one study to address both evaluation 
and research purposes, especially with limited resources. Focus group members stated that a 
study designed to address either one or the other purpose would be hard enough to devise 
and implement. For example, a number of focus group member pointed out the difficulty 



of establishing baseline data and effectiveness criteria for an evaluation of the 
Comprehensive School Program. Similar issues were raised regarding a research study of 
comprehensive school change. Combining the two purposes would add a layer of 
complexity that would undermine the ability of the study to achieve meaningful results. In 
addition, a study designed for both evaluation and research purposes would require more 
resources than a study designed to address either purpose alone. 

Several focus group members commented that if resources were not a consideration, both 
purposes could be more easily incorporated into one study. Nonetheless, it was not clear 
if the problems of conducting such a study could be completely eliminated by increasing the 
resources available. 



2. Study Questions 

Focus group members proposed a range of study questions which could be addressed under the 
evaluation and research purposes. For example, the following evaluation questions were 
suggested: 

(1) What happens in schools that receive Title VII Comprehensive School Program 
grants? What configuration of services do LEP students receive? 

(2) What are the effects of the Title VII Comprehensive School Program funding 
on the services that students receive compared to all the other ways in which 
students receive services? 

(3) If money were spent in an alternative set of ways, would it be better spent? 
Could similar results be achieved without Title VII? 

(4) Have the changes brought about under the Title VII Comprehensive School 
Program grants been successful at providing students with H a meaningful 
instructional experience" as compared to the highest standards of the field at 
the present time? 

(5) Has the Title VII Comprehensive School Program changed the schools' 
perception of how to effectively serve limited English proficient (LEP) 
students? Is there less "marginalization" of programming for LEP kids? 

Participants also suggested the following research questions: 

(1) What are successful models of instruction for LEP students? 

(2) What is the process by which successful models of instruction for LEP 
students were developed and implemented? 

(3) How can greater inclusion of LEP students in the school culture be achieved? 
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One focus group member defined a question which related the evaluation and research purposes. 
"What are the most effective ways to bring money into situations which will leverage the 
kinds of changes which we, as a collective group of educators, think are representative of 
best practices?" By framing the question in this way, the issue of how to best use Title VII 
funds is linked to the larger issue of negotiating purposeful change in schools. 

A somewhat similar set of questions was posed by another focus group member. First, 
"Does the strategy used by the Title VII Comprehensive School Program for bringing about 
educational reform for LEP students work?" And, second, "What is the process by which 
it works best?" In these questions, research and evaluation purposes <.re related because the 
questions focus on the Title VII Comprehensive School Program as one of a number of 
strategies for bringing about change. 

Focus group members did not believe tliat strong causal conclusions could be drawn between Title 
VII funding and outcomes, but they did believe that the study could describe the "footprint" or 
"value added" by Title VII funding. One focus group member asserted, for example, that it 
would be impossible to show that Title VII Comprehensive School Program caused a 
particular change or set of changes within a school. In fact, "you will not be able to attribute 
the source of change in these complex systems to anything." Other focus group members 
agreed that the study of school change should not attempt to attribute school change to 
specific federal, state, or local policies or programs. They believed, however, that such 
policies and programs could be considered as part of the context of a particular site. 

Focus group members discussed the idea of illustrating the "value added" from a Title VII 
Comprehensive School Program grant by describing what has changed within funded 
schools. They noted that the changes that take place could provide evidence of a "footprint" 
of Title VII Comprehensive School Program grants. Focus group members suggested, for 
example, that the study look for changes that are of special importance for LEP students, 
such as improved coherence of instruction, increased validation of student culture, and the 
development of mechanisms for including LEP students in the school community. In 
addition, some focus group members suggested that the extra resources that are provided 
through Title VII Comprehensive School Program grants are at least "fingerprints" of its 
effect in schools. For example, if Title VII funds are used to provide teachers with paid 
release time for planning, that is one effect that can be shown. 

Focus group members indicated that the focus of the study should be both on the content and on the 
process of change. For example, if the research que~ f ion were how to bring about effective 
schools, one approach would be to first define what we mean by an effective or exemplary 
school and then describe the process that the school went through to become more effective. 
Similarly, if the content of change were coherent instruction for LEP students, it would be 
useful to decide, first, the level of instructional coherence within the school and, second, the 
process by which coherence was achieved. Given this approach, it would be worthwhile to 
start with a small sample of schools. Once common processes or themes were identified, 
the study could be expanded to include other schools. 
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One focus group member recommended that the study focus on what happened in individual schools 
since the time a Title VII grant was first received. The study should attempt to understand 
what Title VII has added to the school by focusing on institutional changes that enable LEP 
students to obtain greater access to, participation in, and benefit from classroom and school 
activities. Four levels of variables should be studied: students, classrooms, schools, and 
community context 

More than one focus group member cautioned that a study of change in schools funded under Title 
VII is complicated by the fact that the resources received provide both opportunities and limitations. 
The necessity to comply with federal regulations, for instance, has the potential to influence 
school practices in unforeseen ways. The additional burden of "getting that money, 
spending that money, dealing with how people get access to that money" may affect all 
aspects of the school, including classroom instructional practices. According to one focus 
group member, the instructional approaches emphasized in schools that receive some form 
of federal funding may be less innovative because of the constraints imposed by 
accompanying regulations. 

3. Summary 

• Focus group members recognized the desirability of both the evaluation and research 
purposes of the Benchmark Study. However, they differed in the relative emphases 
they would give to the two purposes. Focus group members expressed concerns 
about whether the evaluation and research purposes could both be adequately 
addressed with the limited resources that they believed were available to OBEMLA. 

• Focus group members proposed a number of evaluation questions concerning the 
effectiveness and costs of Title VII Comprehensive School Grants. They also 
proposed practitioner-relevant research questions concerning best practices for 
serving LEP students. Focus group members felt that OBEMLA needs to prioritize 
the questions on which the study should focus. 

• Focus group members did not believe that direct causal conclusions could be drawn 
about the effects of Title VII funding on LEP student outcomes. They did believe that 
conclusions could be drawn concerning the "footprint" or "value-added" effect of Title 
VII on LEP student services. 



B. General Approach to the Study 



1. Qualitative Versus Quantitative Approaches 

Focus group members empliasized that the study methodologies should depend upon the specific study 
questions to be addressed. The study questions should guide the methodology, and not vice 
versa. 

Focus group members recommended tliat the study include both quantitative (surveys) and qualitative 
(case studies) strategies. They indicated that the exact mix of the two could not be determined 
without definition of the study questions and the amount of resources available for the study. One 
focus group member suggested that the study should take the form of multisite case studies, 
with similar information collected over lime at all research sites. Another member stated 
that case study and survey data can be used to validate each other. This approach, it was 
stated, has not been generally effective in the past, but could be a very strong methodology 
since relating quantitative and qualitative data increases one's ability to generalize results. 
Another member said that findings from quantitative survey data are vastly improved 
through qualitative validation, such as observations of practice and /or intensive interviews 
with teachers and other school staff members. Conversely, even with a strong 
anthropological approach, there is much to be learned by surveying a larger population on 
the topics or themes which emerge from the qualitative data. However, it was pointed out 
that the data collection method should "serve" the question; some questions are simply not 
answerable through one specific methodology, while others can only be. 

2. Core Study Versus Special Topic Substudies 

Focas group members suggested that there should be a "core" study which should remain in place 
throughout the entire study process, but that there should also be flexibility to address new 
questions/issues as they emerge. It was suggested that after examining certain questions in the 
core study, one can address additional issues or relationships in more detail as part of added 
sub-studies. One focus group member indicated that the use of a cooperative agreement as 
the funding mechanism would provide a degree of flexibility for both the Government and 
the researchers. The Government and researchers could work together to redirect activities 
outside of the core to address timely policy issues. 

A broad mix of approaches will be needed, according to one panel member, and there 
should be room for flexibility to modify or add approaches, as appropriate or needed. 

3. Sampling of Title VII and Non-Title VII Schools 

Most focus group members recommended that samples of both Title VII and non-Title VII schools 
should be included in the study. One focus group member noted that only a relatively small 
proportion of LEP students will be served by school-wide Title VII programs. The 
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overwhelming majority will be served in schools which do not receive Title VII support. 
Thus, the study would be remiss if it did not include an examination of non-Title VII 
schools. Most members agreed, saying that you cannot make statements about Title VII 
Comprehensive School Grants without examining and making comparisons with other 
schools. One focus group member stated the idea this way, "You can't tell anything about 
Title VII Comprehensive School Program grants by only studying the schools with them. 
You need to compare them across all kinds of contexts." One focus group member 
suggested that if resources permit, the study should include other schools serving LEP 
students in the same communities as the Title VII schools. 

On the other hand, some focus group members disagreed on whether a comparison group of non-Title 
VII schools was needed in order to draw conclusions about Title VII schools. Several members 
stated that the study should first examine Title VII schools. Three reasons were given: (1) 
a study of funded schools would give OBEMLA a better sense of how well the Title VII 
Comprehensive grant program works — what works and what doesn't; (2) a study focused 
on only funded schools would require fewer resources; and (3) a study that compares 
funded and unfunded schools would increase the complexity of the study without providing 
much additional information. That is, "there will probably be more within group variance 
than between group variance." 

One focus group member strongly recommended that the study only focus on schools 
receiving Title VII funds. She argued that OBEMLA is interested in determining how well 
programs supported by Title VII work, and what can be learned about these programs that 
can inform and improve future efforts. She argued that sufficient resources are not available 
to examine non-Title VII efforts, and she was not sure of what could be learned outside Title 
VII. She stated that sufficient differences exist within Title VII to be able to explore different 
types of programs at schools with different contexts. 

In addition, this focus group member stated that examining a few other programs outside 
of Title VII would not provide sufficient information about how these programs work. One 
would not be able to generalize from the findings, given the variability in how programs 
are implemented across sites. This focus group member stated that instead of including 
schools in the study which do not receive Title VII funds, the study should examine the 
other new Title VII efforts besides the Comprehensive School Grants, particularly the 
program enhancement grants and the system-wide grants. Another focus group member 
generally agreed, but he proposed that the study focus on schools receiving Title VII 
Comprehensive School Grants and examine variation among these schools. The goal of the 
study should be to show how schools receiving these grants work and how they serve LEP 
students. 

Focus group members noted a number of difficulties zvith comparisons of Title VII and non-Title VII 
schools, including considerable variability both within and between programs. For example, it was 
emphasized that "Title VII programs ... are in schools that are systematically different from 
schools that don't comply and compete for Title VII money." And the same program may 
look very different across school contexts. Different local, state and national conditions need 
to be considered when describing how the Title VII Comprehensive School grants work. 
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Nevertheless, one focus group member commented that examining a variety of schools helps 
to describe the "multiple solutions to the problem of serving LEP students." Another focus 
group member stated that an examination of both funded and unfunded schools is the only 
way to answer the question, "Could comparable outcomes be produced without Title VII?" 

4. Size of the Study Sample 

Without knowing the specific study questions to be addressed nor the amount of resources available, 
focus group members could not define how many schools should be studied, A number of focus 
group members expressed a preference for a "deeper" study of a fewer schools rather than 
a "shallow" study of more schools. On the other hand, some participants stated that a 
relatively large number of schools need to be studied so that you can examine them in 
various kinds of local communities and contexts. One member stated that the study should 
examine a wide range of schools with high proportions of LEP students, including equal 
numbers of elementary and secondary schools. In this way, the study could examine the 
effects that other programs, such as Title I and migrant education, are contributing to the 
education of LEP students. Another member stated that it is important to maximize the 
variability of the schools in the sample. This would include Title VII and non-Title VII 
schools. Similarly, any future studies of schools receiving any type of federal funding 
should include schools receiving Title VII funds. 

Focus group members suggested that the number of Title VII and non-Title VII schools included 
should depend upon the specific study .questions and the amount of resources available to the study. 
As one focus group member said, ideally you would study a "gigantic random sample of 
schools." Given a modest amount of resources, however, you would probably emphasize 
a certain set of schools. So, for example, one focus group member believed that the study 
could sample schools that had been shown to be successful in educating LEP students, e.g., 
charter schools. Another focus group member proposed that the focus of the study could 
be on comprehensive school programs, regardless of the funding source. 

Focus group members emphasized that it is important to study specific reforms at multiple schools, 
because reforms operate differently in different settings. The selection of schools for study should 
be based upon ambitious pedagogy, as evidenced by curriculum, instructional approach, or 
assessment technique. A sample of "mature" schools should be selected in which reforms 
have been in place for a number of years, and other schools in which reforms are just being 
introduced. 

One member stated very strongly that once a school was included in the study, it should remain in 
the study for the full data collection period. If additional schools are added in the second or 
third year of the study to address specific relationships or policy issues, these schools should 
be included for the full remaining data collection period. 
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Summary 



The focus group members agreed that the study methodologies should depend upon 
the specific study questions to be addressed. However, the group agreed that both 
qualitative (case studies) and quantitative (surveys) methodologies should be used. 

Participants suggested that there should be a core study which should remain in 
place during the entire study time frame. They also suggested, however, that the 
core study should be supplemented by special issue substudies designed to address 
specific policy or research issues that emerge. 

Most focus group members agreed that samples of both Title VII and non-Title VII 
schools should be included in the study.' They felt that they could not propose 
specific sample sizes without knowing the specific study questions and resources 
available to address those questions. 

Focus group members felt that it is important to study specific types of reforms at 
multiple schools because reforms operate differently in different settings. 
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C Models for Studying the Content of School Change 
1. Content Standards 

There was disagreement as to whether the development and use of standards for judging the content 
of school change efforts was appropriate. The focus group members questioned the use of the 
term "standards" in relation to school reform efforts, and at least one suggested that the term 
"criteria" would be more appropriate. 

Focus group members suggested that it is difficult to define standards related to the content of change 
because such standards reflect the values of a particular group. For example, the effective schools 
literature describes what the classroom learning environment should look like. Yet some 
teachers may disagree with these notions because they are not compatible with their ow ^ 
beliefs about learning. Similarly, standards set at different levels may conflict, e.g., 
standards set by the state compared with standards set by the school. The state, for 
example, may decide that schools should meet standards of ambitious content; at the school 
level, however, that may be "an irrelevant outcome," where, for example, the concern might 
be to first achieve higher student attendance rates. 

It was also suggested that individual differences among students would preclude the development of 
standards. As one focus group member remarked, researchers do not yet know enough 
about what is good instruction for different groups of students — what is a good lesson for 
one set of students may not be best for all kids equally in all contexts. 

Focus group members discussed how one might design a study where the standards are 
based on the local content of the reform — the extent to which schools achieved the goals 
they, themselves, had set. However, there was a concern among at least a few focus group 
members regarding the use of locally-determined standards. One focus group member suggested, 
for example, that a school might decide to set very low standards that could easily be 
accomplished. How would one compare that school with a school where very high 
standards had been set and accomplished? Or even with one where the standards were 
very high, but had not yet been met? 

Focus group members generally agreed, however, that school change could be judged based on the 
extent to which there was: (1) challenging or ambitious academic content for students; (2) coherence 
in the instruction provided; and (3) a climate of collaboration among staff. Focus group members 
defined challenging or ambitious academic content as a climate of high expectations for all 
students indicated by the intellectual quality of the content that is presented to students, the 
mode in which instruction is delivered to students, and the method by which students are 
assessed. Participants noted that there are a variety of different standards' for defining the 
challenge of academic content, but that one way to resolve the problem of different 
standards would be to develop a common set of rules or rubrics relating to the types of 
activities and products in which students are engaged. 
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Coherence of instruction refers to consistency in the way that instruction is provided — a link 
between what is believed about learning, what is taught in the classroom and how it is 
assessed. It includes consistency across different instructional settings and across grade 
levels in a school. Coherent instruction comes about through a shared or common notion 
among school staff regarding appropriate instructional practices. As two focus group 
members noted, there are a number of communication issues that affect the coherence of 
instruction, including the nature of the communication among administrative and teaching 
staff. Whether staff communicate at all — and how well — have implications for the coherence 
of instruction that students receive. 

Focus group members also emphasized a related notion, i.e., the need for "collaboration and 
communication around a shared vision, then a common process for developing" that vision. 
In fact, one focus group member indicated that, regardless of the education goals of a 
particular school, if there is collaboration among staff, the school can be very successful. 

Although they were not as strongly emphasized, focus group members mentioned a variety of other 
guiding principles for assessing the content of school change. Focus group members indicated, 
for example, that the content of change might very well include the extent to which school 
is a safe haven for students, the extent to which school promotes opportunities for shared 
cultural exchanges, and the extent to which school supports the whole student. These 
principles are based on the research on good practices for the education of LEP students. 
The same principles were also related to the focus group discussion of key variables within 
a study of comprehensive school change. 

Focus group members stressed that if standards for the content of change are applied, they should be 
ambitious rather than minimum standards. It was noted, for example, that the standards for 
education that exist at the state and national level are typically minimal, especially the 
standards applied to LEP students. If the intent of comprehensive change is to hold LEP 
students to the same standards as the mainstream, it would be expected that the standards 
for all students be appropriately challenging. At least a few focus group members argued 
that there are reasonably challenging standards within certain disciplines such as science and 
mathematics. And the standards set within Goals 2000 are very ambitious. 

2. Key Content Variables to be Studied 

Focus group members used different categorization systems to define the range of key content 
variables to be studied. One focus group member, for example, suggested the following 
categories for studying variables related to school change: (1) student instructional 
experience; (2) instructional delivery model; (3) staff culture and expectation; (4) nature of 
the change process; and (5) leadership. 

The following system of categorization was also proposed: (1) learning strategies; (2) 
program; (3) school; (4) community contexts; (5) external partners; (6) district; (7) state; and 
(8) federal. 
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A third focus group member discussed a different set of categories: (1) people (students, 
teachers, administrators, parents); (2) activities (language instruction, program coordination, 
decision making, resources allocation, communication between people); and (3) contexts 
(classrooms, schools, communities). 

A fourth member suggested ten categories: (1) student characteristics; (2) student behavior; 
(3) student beliefs/attitudes; (4) teacher characteristics;^) teaching behavior; (6) teachers' 
attitudes /beliefs; (7) classroom structure; (8) classroom social structure; (9) school structure; 
and (10) school social structure. 

The various systems used, however, generally included variables in eight categories. The categories 
were: (1) national, state, district and local context; (2) school structure and characteristics; (3) 
school climate and culture; (4) teacher characteristics; (5) management or governance 
structure within the school; (6) school linkages with parents and /or community; (7) student 
instructional experience; and (8) student characteristics. 

Within each of the eight categories, focus group members provided a list of variables which they 
labelled as key content variables. The list of variables for each category follows: 

(1) National, state, district and local context: 

• federal programs and policies regarding education such as IASA Title VII and 
Title I; 

• state programs and policies regarding education; and 

• demographics of the local community (e.g., income, mobility). 

(2) School structure and characteristics: 

• number of students; 

• grade levels of students; 

• demographic characteristics of the student body (e.g., language, SES); 

• school resources; and 

• resource allocation (e.g., the extent to which school program resources are 
integrated into a coherent program, how resources are allocated for 
professional development for staff who work with LEP students). 

(3) School climate and culture: 

• staff collaboration; 

• the amount of time spent on planning; 

• staff culture and expectations (e.g, the philosophical stance of the 
administration, faculty, and staff regarding the way students learn and 
regarding how LEP students learn; extent to which all staff take responsibility 
for serving LEP students); 

• the relationships among teachers of LEP students and other teachers; 

• the mixture of cultures and languages represented in the school; 

• the extent to which the home culture and language are valued and integrated 
into the school; 

• the primary language spoken in the school; 
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• whether the school is a safe environment for learning; 

• the extent to which the school builds self-esteem and encourages high 
expectations among students; 

• staff development and training (e.g., the extent to which the school develops 
and ensures sensitivity to language and cultural diversity among staff); 

• support services (e.g., library, sports, assemblies, musical performances, plays). 

Teacher characteristics: 

• teaching experience; 

• cultural backgrounds; 

• whether the teacher knows about the culture and/or the personal histories of 
the students; 

• language backgrounds and abilities; 

• teacher beliefs -and attitudes about their own teaching, including their sense 
of responsibility to their students, including LEP students; 

• teacher beliefs and attitudes about language acquisition, collaboration with 
other teachers, and school change; and 

• teaching goals and strategies. 

Management or governance structure within the school: 

• site-based management; 

• accountability systems; 

• teacher participation in decision making and the management of the school; 

• teacher involvement in assessment systems; and 

• the extent to which language minority staff are involved in the decision- 
making process. 

School linkages with parent and/or community: 

• parent and community involvement; 

• the extent to which there is a sense of community in terms of affiliation and 
"commitment toward academic work" among community members; 

• the extent to which the school promotes parent and community involvement 
in school within the minority language population; and 

• the extent to which family health and human services are provided through 
the school to the language minority population. 

Student instructional experience: 

• number of students in the classroom; 

• content of the curriculum ("ambitious pedagogy"); 

• authenticity of the instructional experience; 

• the extent to which there is coherence and coordination of the curriculum and 
instruction over time, and between teachers and developmental levels; 

• integration of services across federal, state, and local programs; 

• teacher behavior regarding LEP students (e.g., the extent to which the teacher 
makes adjustments in instruction that would increase the participation of the 
LEP student); 
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• integration of student culture and language in the classroom (e.g., the 
languages spoken in the classroom, the extent to which cultural diversity is 
valued, the extent to which instruction is compatible with the culture of the 
learner); 

• mode of instructional delivery (e.g., programmatic structure, student grouping 
arrangements, staffing arrangements); 

• assessment measures used; 

• student success in their academic tasks; 

• student engagement in learning (e.g., participation in class and school 
activities); 

• movement patterns in the classroom; 

• the level of student interaction; 

• entry/exit from programs; 

• student perceptions of their safety; 

• student perceptions of a school community; 

• student understanding of what is taught; 

• student academic and social growth (e.g., the extent to which the affective 
needs of students are met); and 

'• equity. 



(8) Student characteristics: 

• culture; 

• education level; 

• home language; 

• self-esteem; 

• attitudes towards teachers and school; 

• attitudes towards other students; 

• family context; 

• socioeconomic status; and 

• other at-risk factors. 



A number of relationships among key variables were emphasized by specific focus group members, 
but there were few common themes across focus group members. For example, the following 
relationships were suggested for inclusion in the study: 

(1) The relationship between change efforts and changes in teacher practice; 

(2) The relationship between school philosophy and instructional practice; 

(3) The relationship between school efforts to develop cultural sensitivity and 
student outcomes; 

(4) The relationship between student culture and student instructional experience; 

(5) The relationship between school resource configuration and change efforts; 

(6) The relationship between teacher decision making and change; 

(7) The relationship between student characteristics and student outcomes; and 

(8) The relationship between teacher strategies and the perceptions of learners 
(e.g., the relationship between the goals of the teacher and the support 
provided to students, and the students' perceptions of their efficacy). 
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Summary 



Focus group members did not believe that standards could be applied for judging the 
content of school change. The use of standards for judging the content of reform 
efforts was seen as problematic due to disagreements among practitioners concerning 
what constitute appropriate goals and effective learning environments for LEP 
students. The use of national, state, or local standards for the purpose of the study 
were all seen to have drawbacks. 

Participants generally agreed, however, that the content of school change could be 
judged based on whether there was ambitious curricular content, coherence of 
instruction, and collaboration among staff. 

Focus group members proposed several systems for categorizing the variables to be 
included in studying the content of school change. In general, however, they 
identified variables in eight categories: 

(1) national, state, and local context; 

(2) school structure and characteristics; 

(3) school climate and culture; 

(4) teacher characteristics; 

(5) management or governance structure within the school; 

(6) school linkages with parents and /or community; 

(7) student instructional experience; and 

(8) student characteristics. 
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D. Models for Studying the Process of School Change 



1. Process Standards 

In general, focus group members indicated that they did not believe that the study should apply 
standards for the process of change. It was suggested, for example, that the process of change 
should be studied empirically rather than theoretically, i.e., the focus should be on what 
changed and how the change was brought about in a particular school. Most focus group 
members agreed that the study should not specify in advance a set process of change that 
schools follow. However, it was suggested that a "core study" examination of how "mature" 
schools have changed could be used to develop models for studying the change process in 
other schools. 

One focus group member, however, specifically advocated the application of standards, and 
emphasized that these should je the same as the standards used to judge the implementation of Goals 
2000. In this way, findings from the study of institutional change can serve as a comparison 
with later studies using the same standards. 

In their comments focus group members did suggest some characteristics of the change process which 
they believed would be related to successful implementation. First, developing a common vision 
among school staff would provide the initial impetus for a particular change effort. 
Evolutionary or gradual change with constant revision and reflection was indicated as a 
second characteristic of successful change processes. One focus group member noted that 
comprehensive change is a sequence of different challenges that must be solved step by step. 
Successful change efforts are those that allow adaptation to changing circumstances without 
losing iight of important /goals. Thus, how decisions are made is important. Third, as the 
change process evolves, it is important to maintain or continue the changes that have taken 
place, i.e., to institutionalize the change so that staff turnover or other changes do not 
undercut the change efforts. A "renewal process" including socialization and training of new 
school staff was suggested as an essential factor in ensuring institutionalization. 

2. Key Process Variables to be Studied 

Focus group members provided sets of organizing principles for studying the process ofcliange. One 
focus group member noted that, although change is not linear and frequent adaptations 
must be made to the content and process of a change effort, there are expected or 
predictable stages that can be documented. The study should include an examination of the 
progress that schools make within these stages and note factors that affect progress at a 
given stage. Another member of the focus group had the idea of setting up a taxonomy of 
the change process in terms of the amount of participation by different individuals, the level 
of conflict, and so on. A third suggested that the change process should be examined at 
three levels: (1) organizational change processes; (2) staff change processes; and (3) student 
change processes. 
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Focus group members also used questions to frame the study of the process of change. One focus 
group member, for example, asked four related questions: (1) Is there ambitious pedagogy?; 
(2) Are there organizational supports for that pedagogy?; (3) How did the pedagogy 
develop?; and (4) How did the supports for the pedagogy develop? A second focus group 
member stated the basic question as, "When a school decides to make a school-wide change 
to include LEP students in all aspects of schooling, what are the kinds of things that they 
have to consider once they begin the change?" 

Focus group members suggested a number of categories of process variables which they thought 
should be examined by the study. These categories were: 

(1) initiation and development of a shared set of change goals; 

(2) development of mechanisms to support change such as professional 
development and resource assistance, e.g., time, technical assistance, training, 
funds; 

(3) planning for change, including setting strategies; 

(4) monitoring/problem solving as the change is implemented; 

(5) adjusting/revising change goals to the situation; 

(6) evaluating the success of the change; and 

(7) institutionalizing the change. 

Focus group members listed considerably fewer process variables than they did content variables. The 
following variables and relationships were suggested: 

(1) The process by which the school reached a shared goal or pedagogy: 

• How did the school start to believe in the importance of educating LEP 
students? 

• How is the school changing to integrate LEP students so they are not 
marginalized and thus not treated differently from other students? 

(2) The process by which the ppropriate organizational structures were put in place to support 
change: 

• How do directives from school and/or district-level leaders get communicated 
to a faculty? 

• What resources or organizational supports were used to facilitate the change 
process and how were they used? 

(3) The manner in which individuals participate in change efforts: 

• Who participates in change efforts, at what stage, how, and how much? 

• What is the pace of change? 

• Is the minority community represented at all stages of the change process or 
only some? In which stages are they represented and what do they do? 

• To what extent are external change agents involved in providing expertise and 
recommendations? 

(4) The persons/factors that facilitate the change process at the various stages; and 
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(5) The persons/factors that prevent the process of change from moving forward at various stages. 



It was noted that patterns of implementing change may be unstable over time as unplanned change 
takes place within the school Turnover of administrative and instructional staff, for example, 
can affect the process of change. Moreover, since change takes time, a study period of five 
years may be too short to develop a complete portrait of the change process. 

3. Summary 

• Focus group members generally agreed that the study should not apply standards 
for judging the process of school change. They believed that the focus should be on 
what changed and how those changes were brought about. 

• Focus group suggested three characteristics of the change process which they 
believed were related to success. These were: 

(1) a common vision among school staff; 

(2) evolutionary change with constant revision; and 

(3) institutionalization of the changes. 

• Participants suggested that the following categories be used in defining the process 
variables to be included in the study: 

(1) definition of goals; 

(2) planning for change; 

(3) mechanisms to support change; 

(4) monitoring and problem-solving; 

(5) revision of goals; 

(6) evaluation of success; and 

(7) institutionalization of the change. 
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E. Methodological Strategies for the Study 
1. Documentation of Classroom Activities 

Most of the discussion relating to the documentation of classroom activities focused on qualitative 
rather than quantitative methodologies. The emphasis was on classroom observation and other 
in-person techniques, though there was some discussion of how teacher questionnaires and 
teacher logs could be used in quantitative studies. 

Focus group members suggested that case study assessments of classroom activities include classroom 
observation, teacher interviews before and after class, examination of curricular materials used, and 
review of actual student work during the class session. One member suggested that after each 
observation, the teacher should be interviewed to confirm their teaching objectives for the 
class session, their intentions, their thought processes, and if and how they are coordinating 
with other teachers. One member said that his experience shows that it takes a lot of time 
talking to and observing teachers to be able to understand what strategies teachers use to 
serve LEP students. It was stated that, researcher-teacher conversations are just as important 
as observations. Another focus group member suggested that data collection approaches 
include teacher and student interviews, teacher and student focus groups, telephone 
interviews with teachers, classroom observations, and observations of individual students 
from whom classwork is collected on a regular basis over time. One focus group member 
suggested the used of video cameras, but most participants felt that the use of video cameras 
would be too intrusive. 

It was generally agreed that classroom observation is a very expensive methodological approach. 
Nevertheless, focus group members believed that classroom observation was extremely important. 
Classroom observation requires considerable resources in order to be done well. 
Observations and the analysis of the data which are collected would be fairly intensive with 
respect to the resources required. One focus group member pointed out that because 
qualitative data are very difficult and time-consuming to collect and analyze, researchers 
should plan in advance how they plan to combine the qualitative and quantitative data. 
Another focus group member suggested that, since classroom observation is very expensive, 
it could be done on relatively few occasions during the first couple of years of the study, 
and then more frequently during the later years. 

Focus group members generally agreed that classroom observations should occur a number of times 
over the school year in order to capture change in the learning environment and teacher-student 
relationships. All subject areas would not necessarily be given equal weight, or even 
included. The study could focus on a few specific subject areas. Focus group members also 
suggested that standardized protocols should be used to help observers capture and record activities 
in the classroom, and so that cross-site analyses could be performed. 

Focus group members stressed the importance of using classroom observers who are knowledgeable 
about the content area being taught and ivho are proficient in the language of instruction. One 
member pointed out that case study researchers should be bilingual and multicultural and 
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assigned to sites based on their knowledge of the home culture of the students in the school. 
They should be trained to be skilled observers. Another focus group member stated that 
highly trained research teams, with relevant language capabilities, are needed and that they 
need to spend lot of quality time in the schools. 

Focus group members urged that pre-class teacher interviews include discussion of the goals and 
objectives of the class period and of the strategies to be employed. Post-class interviews would'include 
discussion of goal achievement and reasons why specific classroom activities took place. Teachers 
should be asked to indicate those sections of textbooks and other curricular materials which 
are the focus of instruction. In addition, information on planning activities and other 
professional activities engaged in by teachers should be obtained. 

Focus group members stressed that a key variable that should be assessed at the classroom level is the 
"academic rigor" of the material that is being presented. They suggested that the national 
standards which are being developed provide some guidance in making these assessments. 

For the quantitative component of the study, focus group members suggested the use of teacher 
questionnaires and teacher logs to study classroom activities. They suggested that there were a 
number of items from the NELS teacher questionnaires which had been shown to be useful 
in describing instructional practice, on topics such as the mode of instructional delivery and 
participation in governance. Participants also indicated that investigators had successfully 
used teacher logs to collect data on classroom activities.. 

2. Measurement of Student Outcomes 

Focus £ mp members disagreed on the extent to which the study should focus on student outcomes. 
Some of the focus group members warned that student outcome data are difficult and 
expensive to collect. It was noted, for example, that there are very few appropriate student 
outcome measures at either the local, state or federal level, especially for limited English 
proficient students. One suggestion was to compare the content of instruction within a 
particular school against national content standards (Goals 2000) and determine whether or 
not the standards had been met. 

Most focus group members did not believe that the cost of developing and validating new student 
achievement tests was justified for this study. One focus group member did recommend the 
design of new instrumentation. He stated that this would be the ideal approach, and in that 
manner you would have the same measures at each site that are valid and reliable. 

Focus group members raised concerns about realistically assessing student outcomes in relation to 
the changes brought about through the Title VII Comprehensive School Program given the nature 
of the change process and the relatively short time period of the proposed study. They raised the 
question of what kind of gains one could reasonably expect to see over a five-year period. 
Focus group members further cautioned that a focus on student outcomes brings in a whole 
set of complicating factors. For one thing, the various contexts within which services are 
received must be considered, including classroom, school and district. One focus group 
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member described how the "collective expectations and practices" of individual schools have 
been shown to affect student outcomes across years so that over and over, different sets of 
students achieve at a similar level. Describing the collective expectations and practices of 
a particular school and keeping these factors in mind increases the complexity of the study 
design and, of course, the time and costs involved. One focus group member stated that to 
do an outcome study that considers all the contexts in "sensitive ways" would cost a 
minimum of a million dollars per year. 

A few focus group members emphasized that, despite the difficulties of designing and implementing 
a study focused on student outcomes, such assessments are important. In fact, at least one focus 
group member remarked that the true relevance of any change effort is student outcomes, 
i.e., how it affects an individual student in terms of the configuration of services they receive 
and their task-related performance. At least one focus group member stressed the 
importance of choosing between the measurement of outcomes and the measurement of 
processes. The focus group member noted, for example, that given the range of contexts, 
"even under the most ambitious of thoughts about funding levels, you can't do both." 

Focus group members generally agreed tliat there are not yet clear national standards against which 

to judge the quality of student work. However, it was the consensus of the group that samples of 

student work should be collected and scored against some generic rubric. Separate scales will need 

to be developed for content and for language. The issue of valid assessment of LEP students 

is a major issue — what are the standards against which LEP students should be judged? 

Sufficient resources will be needed to develop appropriate indicators and rubrics for 

measuring student outcomes. It was agreed that there is a lack of appropriate measures of 

LEP student performance. One focus group member suggested that data on grades, 

portfolios, and other assessment instruments should be collected at the end of each year. s 

Focus group members urged that student work products from the classroom should be examined in 
order to judge if that work is of "rigorous " quality. One focus group member suggested that the 
study examine the quality of the content presented to students along with the performance 
of students with respect to that content. The member stated that examining instructional 
content "and student performance is difficult to do for more than a limited number of 
lessons, and thus it would be difficult to determine what is taught and learned over time. 

In examining outcome data for LEP students, focus group members stressed that serious efforts need 
to be made to separate the variance due to content area knowledge from the variance due to language 
proficiency. One focus group member suggested that measurement of content versus 
language can be disentangled through the design of items in different languages which 
replicates similar content. Another member said that testing of LEP students will be a 
problem since many schools, especially "progressive" schools which have tried to implement 
an ambitious curriculum have an ideological aversion towards testing in general, and 
especially of LEP students. 
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3. Measurement of Other Variables 



Focus group members suggested a range of methodologies which could be used to address 
variables such as governance /decision-making, instructional planning, communication 
within the school, and linkages with parents and the community. Participants suggested that 
small group interviews with students should be used to assess their perspectives on student-teacher 
relationships, their general views of the school, and their views on their instruction, themselves, and 
their schoolmates. Focus group members said that students as low as Grade 3 could be 
included in such interviews. Students should also be observed in non-classroom settings such as 
libraries, hallways, cafeterias, etc. 

Participants also suggested that in addition to observing teachers in classrooms 
methodologies should include observing them in planning, departmental, and management 
meetmgs. Interviews with principals and focus group sessions with teachers and with 
parents would also be useful. 

For the quantitative component of the study, participants suggested the use of teacher questionnaires 
to examine such factors as teacher collaboration and school climate. Items from the NELS teacher 
questionnaires were again* cited as useful models on these topics. 

One focus group member pointed out that self-reports of attitudes and beliefs are extremely 
unstable. Thus, self-reports need to be anchored in behavior and actual activities. Questions such 
as Do you hke school?" need to be followed up by asking about what specific things you 
like to do at school. Also, participants stressed the importance of using multiple measures 
of the same variables in order to increase the reliability of the data. In particular they 
suggested that findings from qualitative analyses be validated through quantitative 
assessment. ° n 

4. Recruitment of Sites for the Study 

Focus group members stressed the importance of incentives to gain the cooperation of school 
organizations and school staff for the study. The following types of incentives were suggested: 

• Paying substitute teachers during site visits to record data or to take over 
classes for regular teachers; 

Paying regular teachers an honorarium for their time devoted to the study; 

Paying overtime for school staff to come together for meetings, conferences, 
etc. related to the study; and 

• Providing funds to schools to pursue their own research interests. 



25 



I 



I 
I 

I 
I 
I 
I 
I 



I 
I 
I 
1 
I 
I 
I 



5. The Role of Investigators 

One focus group member suggested four possible roles for investigators of a change process. These 
four roles fall on a continuum from high involvement of investigators in which the change 
process is research-directed, to low involvementin which the change process is practitioner- 
directed. The four roles are: 

• Planned variation model in which developers implement an explicit model for 
educating LEP students at multiple sites, and also perform the evaluation; 

• Organizational development model in which an external change agent 
supports the development of school-initiated education models and studies the 
implementation process; 

• Evaluation /feedback model in which researchers provide feedback to 
practitioners on results; and 

• Non-involvement model in which researchers gather data but do not share 
results with practitioners. 

In order to maintain a positive working relationships, focus group members believe that investigators 
need to share some of their observations with school staff. One member stated that interactions 
between researchers and their subjects typically result in exchanges of information, such as 
providing information on references to check, contacts to communicate with, or informal 
reflections on instructional and teaching practices. These exchanges may be informational, 
but not directive. Another focus group member said that it is unavoidable that researchers 
will not be involved at least indirectly in the change process. Ideas get passed from one 
person to another as researchers interview or talk informally with school staff. 

On the other \\and f focus group members do not believe that investigators should be active agents of 
school change. One member said that this creates difficult role conflicts. Another focus group 
member said that investigators should not be involved in the change process while the study 
is underway. Another member said that the researchers should not be directly involved in 
planning changes or in telling schools what to do. They may offer suggestions or ideas in 
the course of holding debriefing sessions. People will learn from these discussions and may 
implement some of these suggestions. But the researchers should not be any more directly 
involved. 
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Summary 



Focus group members emphasized the importance of classroom observation. They 
believed that teachers should be observed several times over the course of a school 
year by trained observers with relevant language capabilities using standardized 
protocols. 

Participants also believed that in case studies: 

(1) teachers should be interviewed to determine their objectives and teaching 
strategies; 

(2) curricular material should be examined to determine the challenge of the 
content; 

(3) student work should be reviewed to determine student achievement of 
curricular objectives; 

(4) students should be interviewed in small groups to obtain their 
perspectives on instruction, student-teacher relationships, themselves, 
their classmates, and their school; and 

(5) students should be observed in non-classroom settings such as libraries, 
cafeterias, and hallways. 

Focus group members disagreed on the priority which should be given to the 
measurement of academic achievement by LEP students. There were concerns about 
the ability to provide evidence of change which could be directly attributed to Title 
VII, given the large number of factors operating in a school environment, the slow 
pace of school change, and the time frame of the study. 

Focus group members indicated that there are presently no clear national standards 
against which to judge the quality of student work. However, they believe that the 
study should use emerging national standards to create scales and indicators which 
can be used to make such judgments. These scales and indicators will need to 
separate the effects of content area knowledge and language proficiency. 

To gain the cooperation of school staff, the study should consider paying substitute 
teachers to collect data, paying teachers and other school staff for the time they 
devote to the study, and providing funds to the school to pursue its own research 
interests. 

Focus group members do not believe that the study investigators should participate 
directly as active agents of school change. However, the participants recognized, that 
in order to maintain cooperative relationships, the investigators need to share some 
of their observations with school staff. 
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IV. RECOMMENDATIONS 



This section presents recommendations from Development Associates, Inc. concerning the 
design of a study of school change involving LEP students. The recommendations are based 
primarily on the oral and written comments of focus group members, but also reflect our 
work in preparing the literature review which was used as a resource document for the 
focus group, and experience in conducting other national studies of LEP services. The 
recommendations are organized under the topic headings used in the report, and each 
recommendation is referenced to the relevant pages of discussion in the findings section. 

Study Purposes and Study Questions 

• OBEMLA should define the specific questions to be addressed by the study. These 
questions should expand on the questions suggested by focus group members, and 
should reflect the relative emphases which OBEMLA wishes to give to the evaluation 
and research purposes of the study, (p. 5-8) 

• The study should explicitly focus on both the content of school change and the 
process of school change. In order to be useful to the field, the study needs to define 
both what interventions are effective in improving services to LEP student* and how 
those changes were implemented. The study should also describe the contexts in 
which specific reform efforts are effective, (p, 7) 

• The study should not attempt to causally link Title VII resources to specific classroom 
and student outcomes. The study can attempt to show how Title VII resources were 
used to expand or improve services for LEP students, (p. 7) 

General Approach to the Study 

• The study should include both quantitative and qualitative components. The 
qualitative (i.e., case study) component will add depth to the analysis, while the 
quantitative (i.e., survey) component will add breadth to the conclusions, (p. 9) 

• Special care should be taken in linking the qualitative and quantitative components 
of the study. This can be done by developing survey questions based on case study 
results, by using similar terminology in surveys and case study protocols, and by 
analytically linking classroom and student outcomes observed in case studies with 
specific interventions identified in both case studies and surveys, (p. 9) 

• The design should include both a "core" study to be defined at the beginning of the 
process, and "special topic" substudies to be added as new and interesting research 
issues are identified. OBEMLA and the researchers should agree on the amount of 
resources to be reserved for special topic substudies, and should have regular 
meetings to decide on the implementation of those substudies. (p. 9) 
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The study should include both schools receiving Title VII funds and schools not 
receiving such funds. The relative numbers of schools in the two categories should 
depend on the relative emphases given to the evaluation and research purposes of 
the study, (p. 9-10) 

To the extent possible, case studies should be focused on schools which have either 
demonstrated success in serving LEP students or show promise for doing so. 
Random selection of case study schools is likely to produce too many examples of 
mediocre or poor services and outcomes, (p. 11) 



Models for Studying the Content of Change 

• The researchers should apply a comprehensive and coherent model for defining the 
content of school change efforts. This model should reflect both the goals of -the 
change effort and the area= of focus (e.g., school climate, instructional methods, etc.) 
of the effort, (p. 13-14) 

i 

• The model for the content of change which is applied should give prominence to the 
level of challenge which is offered to LEP students, the coherence of the instruction 
provided, the level of communication /collaboration among school staff and with the 
community, and the level of concern shown for the whole student, (p. 13-14) 

• There should not be universal criteria and standards against which the contents of 
school-level change efforts should be judged. Schools should be judged based on 
success in achieving their own content goals. They should also be judged based on 
whether LEP students are mastering challenging academic content, (p. 13-14) 



Models for Studying the Process of Change 

• The researchers should apply a coherent model for defining the process of school 
change efforts. This model should describe steps or stages in the change process as 
well as indicating key elements of the change process (e.g., context, participants, 
strategies, etc.). (p. 19-21) 

• The model for the process of change which is applied should give prominence to the 
strategies for change which are used (e.g., development of a school-wide vision, 
changing staff attitudes, etc.), the organizational outcomes which are expected, and 
the student outcomes which are expected to result, (p. 19-21) 

• There should not be universal criteria and standards against which the process of 
school change efforts should be judged. Schools should be judged based on whether 
the planned process can be expected to achieve the school change goals, (p. 19) 
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Methodological Strategies 

• The quantitative component of the study should include teacher questionnaires 
assessing such variables as instructional approaches used, attitudes towards LEP 
students, and expectations for those students. School-level questionnaires should also 
be used to define the specific content and process of school change efforts. To the 
extent possible, attitude questions should be related to specific behavioral outcomes 
(e.g., What percentage of the LEP students in your school do you expect to attend 
college?), (p. 23, 25) 

• The qualitative component of the study should include classroom observations, 
teacher interviews before and after observations, interviews with LEP students in 
small groups, examination of curricular material used, and review of student work 
to determine success in mastering ambitious content. The qualitative component 
should also include observations of students within school settings other than 
classrooms, observations of staff planning sessions and other meetings, and focus 
groups with school staff and with parents and community members, (p. 22-23, 25) 

• The study should not attempt to develop or use a consistent set of LEP student 
achievement measures across sites. Existing standardized tests are not appropriate 
for this purpose, and national and state standards have not been developed to the 
point where new tests can be developed, (p. 23-24) 

• The researchers should be urged to use as resources the emerging national standards 
in order to develop approaches for assessing the level of challenge of instruction 
being offered to LEP students, and the students' mastery of the material, (p. 23-24) 

• The researchers should be urged to develop and use a consistent set of language 
proficiency measures across sites. The measures should include teacher ratings of 
oral proficiency and literacy in English and, to the extent possible, the native 
language, (p. 24) 

• The researchers should not be involved in the design or implementation of school 
change efforts. To maintain a positive relationship with sites, however, case study 
researchers should in a nonjudgmental fashion share their observations about the 
process of change in the school sites, (p. 26) 
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Focus Group Participants 

Schedule and Agenda for the Focus Group Meeting 
Written Recommendations from the Participants 
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SPECIAL ISSUES ANALYSIS CENTER 

Focus Group on Measuring Institutional Change 
Affecting LEP Students 

/ Agenda 

Wednesday, Tanuary 11 

A.M. 

8:30-9:00 Coffee and Informal Greetings 

9:00-9:10 Welcoming Comments 

Annette Zehler, Development Associates 

9:10-9:25 Purpose of the Meeting 

Eugene Garcia, Director, OBEMLA 
Timothy D'Emilio, OBEMLA 
Millie Lanauze, OBEMLA 

9:25-9:30 Summary of the Schedule of the Meeting 

Paul Hopstock, Development Associates 

9:30-10:50 Topic 1: Research Questions for a Study of School Change 

10:50-11:00 Break 

11:00-12:30 Topic 2: Models for Studying the Content of School Change Efforts 

P.M. 



12:30-1:30 Lunch (Conference Room) 

1:30-2:00 Topic 2: (continued) 

2:00-3:00 Topic 3: Models for Studying the Process of School Change Efforts 

3:00-3:10 Break 

3:10-3:45 Topic 3: (continued) 

3:45-4:45 Written Recommendations Relating to Topics 1-3 

4:45-5:15 Final Comments and Reactions on the Day's Topics 
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Thursday, January 12 

A.M. 

8:30-9:00 Coffee and Conversation 

9:00-9:10 Overview of Day 2 Activities 

9:10-10:30 Topic 4: Methodological Approaches to Studying School Change 

10:30-10:40 Break 

10:40-12:00 Topic 4: (continued) 

P.M. 

12:00-1:15 Lunch (Reservations at Local Restaurant) 

1:15-2:30 Topic 5: Methodological Issues in Studying School Change 

2:30-2:40 Break 

2:40-3:30 Topic 5: (continued) 

3:30-4:30 Written Recommendations on Topics 4 and 5 

4:30-5:00 Final Comments and Reactions to the Day's Topics 
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SPECIAL ISSUES ANALYSIS CENTER 



Focus Group on Measuring Institutional Change 
Affecting LEP Students 



Topic 1: Research Questions for a Study of School Change 

(a) To what extent should the study focus on all schools serving LEP students as 
opposed to schools receiving Title VII comprehensive school grants? 

(b) To what extent should the study focus on a before-and-after examination of 
school change as opposed to an examination of the process of change itself? 

(c) To what extent should the study attempt to attribute school change to specific 
federal, state, or local initiatives or programs? 



MC 
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SPECIAL ISSUES ANALYSIS CENTER 



Focus Group on Measuring Institutional Change 
Affecting LEP Students 



Topic 2: Models for Shidying the Content of School Change Efforts 

(a) Should there be standards relating to the content of educational reform against 
which schools should be judged? If so, what are the key elements of those 
standards? 

(b) What are the major categories of variables that should be studied relating to 
the content of educational reform? What are the key variables within each of 
those categories? 

(c) What relationships among categories of content variables or among individual 
variables should receive particular attention in the study? What are the 
theoretical models which suggest the importance of such relationships? 



SPECIAL ISSUES ANALYSIS CENTER 



Focus Group on Measuring Institutional Change 
Affecting LEP Students 



Topic 3: Models for Studying the Process of School Change Efforts 

(a) Should there be standards relating to the process of school change against 
which schools should be judged? If so, what are the key elements of those 
standards? 

(b) What are the major categories of variables that should be studied relating to 
the process of school change, including factors which promote or impede 
change? What are the key variables within each of those categories? 

(c) What relationships among categories of change variables or among individual 
variables should receive particular attention in the study? What are the 
theoretical models which suggest the importance of such relationships? 



17o 



SPECIAL ISSUES ANALYSIS CENTER 



Focus Group on Measuring Institutional Change 
Affecting LEP Students 



Topic 4: Methodological Approaches to Studying School Change 

(a) Should the study employ a broad mix of methodological approaches, or 
should it focus on the strong application of one or two approaches? 

(b) What specific approaches should be employed? If a mix of approaches is 
suggested, how much of the study's resources should be devoted to each 
approach? 

(c) What specific approaches are recommended for collecting data over time on 
teaching /learning environments and teacher/learner relationships? 

(d) What are the design requirements of the methodologies which are 
recommended (i.e., how many measurements of what type with whom and 
over what time period are required)? 

(e) What is the recommended role of investigators in the change research in terms 
of objectivity and involvement in the change process? Should they make 
recommendations and /or be involved in implementing changes? 



JC 



176 



SPECIAL ISSUES ANALYSIS CENTER 

Focus Group on Measuring Institutional Change 
Affecting LEP Students 

Topic 5: Methodological Issues in Studying School Change 

(a) What are the key issues which complicate the design of the study (e.g., 
sampling, number of measurements)? How can the study best address those 
issues? 

(b) What are the key issues which threaten the validity of measurement in the 
study (e.g., the level of measurement, appropriateness of measures for LEP 
students)? How can the study best address those issues? 

(c) What are the key issues which will complicate the analyses of the study data 
(e.g., the presence/absence of standards for assessing change, difficulty in 
drawing causal inferences)? How can the study best address those issues? 



ERLC 
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Topic 1: Research Questions for a Study of School Change 
Written Recommendations from: Betty Mace-Matluick 

(a) To what extent should the study focus on all schools serving LEP students as 
opposed to schools receiving Title VII comprehensive school grants? 

The sample for the study should include both Title VII and non-Title VII schools 
since the majority of LEP students will be served in schools that do not have Title VII 
funds. Most of the LEP students, in both Title VII and non-Title VII schools, will be 
in schools where Title I funds, state bilingual, and other special funds are targeted 
.. because of the SES level of LEP students in general. Another consideration is that 
the level of Title VII funding now (and probably into the future) is such that an 
extremely limited number of school grants can be awarded each year... and 
undoubtedly those schools who apply for funds and receive funds will be atypical 
of schools that enroll LEP students. Therefore, I believe that more can be learned 
about whether on not LEP students are better served with the comprehensive 
schoolwide reform strategy than under the previous strategy of categorical program 
funding at the federal level. Also, by including a broader sample, the study could 
look at whether or not additional resources (i.e., Title VII funds) tends to stimulate, 
accelerate, and /or sustain (or have some negative effect??) on educational 
restructuring at the school level. 

(b) To what extent should the study focus on a before-and-after examination of school 
change as opposed to an examination of the process of change itself? 

The study should document where a given school is in the process of change related 
to LEP students at entry into the study and follow and study the process and its 
outcomes (both in terms of organizational change and outcomes and student 
experiences and outcomes). I would be more interested in the effects on the change 
process as the school moves toward inclusion of LEP students into the life of the 
school, as a result of additional resources, than on outcomes in general ... because I 
think that it is assumed that LEP students will be better served in comprehensive 
school wide programs than in other program options... and this assumption should 
be evaluated ... 



(c) To what extent should the study attempt to attribute school change to specific 
federal, state, or local initiatives or programs? 

The study should attempt to examine all factors that influence change, including 
external resources that the school itself solicits and /or initiates (e.g., Title VII). 
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Topic 1: Research Questions for a Study of School Change 
Written Recommenadations from: Brian Rowan 

(a) To what extent should the study focus on all schools serving LEP students as 
opposed to schools receiving Title VII comprehensive school grants? 

I believe the study should examine Title VII comprehensive AND other types of 
schools, the purposes of the study are twofold: 

1. Of those schools that receive Title VII comprehensive grants - what outcomes 
were produced 

2. Could comparable outcomes be produced without the Title VII grants process. 

(b) To what extent should the study focus on a before-and-after examination of school 
change as opposed to an examination of the process of change itself? 

The study needs to have before data on key variables, should collect data at several 
more points in time, to establish dynamics of change, and should attempt to explain 
these data by reference to how change was produced. 

(c) To what extent should the study attempt to attribute school change to specific 
federal, state, or local initiatives or programs? 

I doubt that one can attribute change to a particular source. Change pt obably results 
from a combination of factors which may be unique to each site. 



Topic 1: Research Questions for a Study of School Change 
Written Recommendations from: Christian Faltis 



(a) To what extent should the study focus on all schools serving LEP students as 
opposed to schools receiving Title VII comprehensive school grants? 

I think that the bulk of the study should focus on title VII comprehensive schools 
receiving grants, and the study should be prepared to look for intra-group variation. 
There probably should be a number of non-Title VII schools serving LEPs involved, 
but without an expectation of meeting requirements for a quasi-experimental design. 
The goal of the study should be to show how TVII comprehensive schools work, and 
how they serve LEP students. There may be some comparison with non-TVII 
schools. 

(b) To what extent should the study focus on a before-and-after examination of school 
change as opposed to an examination of the process of change itself? 

I think the study should focus primarily on the process of change to show what 
schools are doing as they reform to include LEP students. Part of this study of 
process would of course entail looking at what happened before and after the change 
was being planned, initiated, and institutionalized (if it gets this far.) 

(c) To what extent should the study attempt to attribute school change to specific 
federal, state, or local initiatives or programs? 

In think that in the overall plan of study, federal, state, and local initiatives have to 
be considered as key variables, along with other content variables, such as student 
learning, school organizing, program changes, teacher development, and community 
participation. 

Accordingly, I think it is important to include in the study schools that have not 
received TVII funding, but are attempting reform to include LEP students 
nonetheless, through other sources of financial and pedagogical support. 
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Topic 1: Research Questions for a Study of School Change 
Written Recommendations from: Charlene Rivera 



(a) To what extent should the study focus on all schools serving LEP students as 
opposed to schools receiving Title VII comprehensive school grants? 

Insofar as possible, the study should examine a wide range of situations with high 
proportions of LEP students. Careful consideration should be given to including at 
least equal numbers of elementary and secondary programs. The study should 
include an examination of how various program components, including Title 1, 
migrant, etc., are contributing in positive ways to the education of LEP students as 
measured by student outcomes. 

(b) To what extent should the study focus on a before-and-after examination of school 
change as opposed to an examination of the process of change itself? 

Both need to be included. Without an examination of process there will be little 
information about when and how change has occurred overtime. If resources are 
limited, fewer sites should be studied rather than simply include baseline and 
outcome data. 

(c) To what extent should the study attempt to attribute school change to specific 
federal, state, or local initiatives or programs? 

In the new conception of Goals 2000, monies from a variety of sources will be 
intermingled and used to provide support to LEP students. Given this reality, it 
would seem contradictory to try to attribute particular changes to any one program. 
Whatever is learned will need to be understood as part of a comprehensive effort 
within a whole school /district program. 



Topic 1: Research Questions for a Study of School Change 
Written Recommendations from: Diane August 

(a) To what extent should the study focus on all schools serving LEP students as 
opposed to schools receiving Title VII comprehensive school grants? 

First, I think the major reason for focusing on school that do not receive Title VII 
school grants would be to compare the performance of schools receiving Title VII 
comprehensive school grants with the performance of schools that do not receive 
these grants. 1 One would randomly sample schools from each groups and compare 
their performance on a predetermined set of outcome variables. 

I would argue against this approach for several reasons. First, there will probably 
be more within group variance than between group variance (especially given the 
regulations that define these programs that provide maximum flexibility). As such 
there will be no significant differences between groups. Second and more important, 
even if there are significant between group differences it would be very hard to 
explain to what they are attributable — given the tremendous variation across sites in 
how those programs will be run and in the context in which they operate. 

As such, I would argue for avoiding the issues of Do these programs work better 
than XX other programs and focussing instead on whether students in these 
programs are learning challenging subject matter and skills, what is happening in the 
schools that help students accomplish this and what impediments, challenges exist 
for the school that do not. I would be careful to look at differences among 
subgroups of students in each school and to be able to compare across sites. 

(b) To what extent should the study focus on a before-and-after examination of school 
change as opposed to an examination of the process of change itself? 

The study should examine both. The challenge will be to figure out what is 
happening that is new as a result of Title VII grant money and requirements. This 
will require looking at before characteristics. However, to get a good sense of how 
the projects are implemented it will be necessary to look at the change process as 
well as outcomes. 



Another reason for looking at school that are not Title VII comprehensive schools 
would be to learn about exemplary practices that might not be taking place in the 
comprehensive schools. I see no point in doing this in that one would only learn 
about what is happening in the observed schools and really could not generalize to 
other schools, either Title VII comprehensive schools or other non Title VII schools. 
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To what extent should the study attempt to attribute school change to specific 
federal, state, or local initiatives or programs? 

The study should not attempt to attribute student outcomes to federal, state, and 
local initiatives, but might look at how these initiatives have impacted the school in 
terms of such things as appropriateness of curriculum, program management, staff 
development, and coordination. This is a complex matter because school change 
comes about for many reasons, some of which have nothing to do with outside 
initiatives. 
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Topic 1: Research Questions for a Study of School Change 
Written Recommendations from: Julia Smith 

(a) To what extent should the study focus on all schools serving LEP students as 
opposed to schools receiving Title VII comprehensive school grants? 

(b) To what extent should the study focus on a before-and-after examination of school 
change as opposed to an examination of the process of change itself? 

(c) To what extent should the study attempt to attribute school change to specific 
federal, state, or local initiatives or programs? 

Abandoning all the above questions, I suggest that the question of this topic is, "What 
is the research question(s) for the RFP?" I would recommend as response to this 
topic the following interrelated questions: 

1) How does Title VII work? 

2) Under what circumstances are LEP students best served? 

Part 1 addresses the character of services provided in schools, using student 
performance measures, organizational management measures, and institutional 
change measures as benchmarks against which one assesses the functioning of those 
services. In other words, this part addresses WHAT the service is, HOW it is 
implemented in the school, HOW the school delivers the service, and ultimately 
HOW WELL the school can be said to be serving the needs of LEP students based 
on a comparative standard. 

Part 2 addresses the experiences of students receiving what is likely to be a myriad 
of services, and focuses on the individual impact of amount and kind of services on 
relevant outcomes -- i.e., achievement, engagement, view of self, understanding of 
citizenship, etc. 

This part addresses the student as an active agent - receiving, processing, and 
producing within a wide variety of contexts which all may be present in a given 
school. 
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Topic 1: Research Questions for a Study of School Change 
Written Recommendations from: Judith Torres 

(a) To what extent should the study focus on all schools serving LEP students as 
opposed to schools receiving Title VII comprehensive school grants? 

In the best of all possible worlds, the study should maximize the variability of 
schools in the sample. This would imply including schools other than just those 
receiving Title VII funding. Similarly, any planned studies of schools receiving other 
federal funding should include schools being funded by Title VII. 

(b) To what extent should the study focus on a bef ore-and-after examination of school 
change as opposed to an examination of the process of change itself? 

This should probably vary, depending on the particular study subquestion to be 
addressed. An examination of the change process itself should preferably be 
conducted at relatively frequent intervals during the academic year. Changing 
patterns of interaction, changes in understandings which emerge from collaboration 
and communication, need to be documented at several points. Collection of other 
information (outcome variables, for example) could be done at less frequent intervals. 
On the other hand, analysis of the outcome information may be very time 
consuming, particularly if the researchers decide to analyze samples of student work. 

(c) To what extent should the study attempt to attribute school change to specific 
federal, state, or local initiatives or programs? 

It is highly unlikely that a study can attribute change to specific programs. I think 
what it can do is examine the changes which occur where different resource 
configurations may be found, and funding source information will be key in this 
regard. Some combinations of funding sources may prove synergistic, while others 
may not. One major question is what happens to the use of Title VII funding in the 
presence or absence of other resources. Do services for LEP students remain salient, 
or are they submerged in general efforts? Or, in contrast, where other resources are 
lacking, are Title VII resources sufficient to support school change? 

Because funding sources provide a resource base for much innovation at the school 
level, documentation of available resources will be important. 
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Topic 1: Research Questions for a Study of School Change 
Written Recommendations from: Paul Berman 



The study should address the following questions: 

1. What impact has Title VII had on how schools and districts deliver education 
to (and provide other services for) LEP students and how have these changes 
affected student learning and other goals? 

2. What school-wide "approaches" to educating LEP students are effective, under 
what "conditions"? The term "approaches" could be called principles or 
strategies or other terms. It encompasses the systemic (and school-level) 
interrelationships among teaching/learning strategies (which includes 
curriculum and instruction; the programming for language acquisition (which 
includes English-language acquisition in written and oral forms); the structure 
and culture of schooling (for all students with LEP students as part of the all); 
the linkages between school and community, school and district, and 
educators and their peers outside the school. The term "conditions" is taken 
to mean the demographic, political, economic, cultural and capacity situations 
of the school. There is no single approach or configuration of approaches that 
is effective in all settings. Therefore, the research should aim to identify what 
is effective for different conditions that encompass a reasonable range of 
critical variation. 

3. How did the effective school-wide approaches happen, and are there any 
general patterns in the process of developing effective approaches? 

4. How can schools (with LEP students) become more effective? The answers 
to this question are the policy and practice implications of the findings from 
points 2 and 3 above. 

5. How can federal policy in general and IASA in particular be improved? The 
ansv/ers to this question are based on answers to all the above questions. 

(a) To what extent should the study focus on all schools serving LEP students as 
opposed to schools receiving Title VII comprehensive school grants? 

The study should first study schools receiving Title VII grants. If resources permit, 
the study would be much more valuable - that is, it could better address each of the 
above five questions - if it include a sample of other schools serving LEP students. 
The notion of a benchmark would be more meaningful. However, these schools 
should not be studied if resources are not available. The proposers should offer a 
design to deal with this tradeoff. 
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To what extent should the study focus on a before-and-after examination of school 
change as opposed to an examination of the process of change itself? 

The study should examine the process of change across a range of levels which focus 
on institutional change as manifested in student learning, curriculum and instruction, 
school organization and structure, and community linkages. This does not preclude 
doing a before-after design as an element of the research. 

To what extent should the study attempt to attribute school change to specific 
federal, state, or local initiatives or programs? 

The study should examine the perceived and actual effects of intentional policies at 
all levels. However, the causation can never be clear so that the "findings" need to 
be qualified. 



Topic 1: Research Questions for a Study of School Change 
Written Recommendations from: Walter Secada 



(a) To what extent should the study focus on all schools serving LEP students as 
opposed to schools receiving Title VII comprehensive school grants? 

This question, really is a methodological issue. The question as to what sorts of schools 
to sample should be driven by your research questions and/or the political 
constraints of the study. For instance, if you are limited to studying Title VII 
grantees because of congressional mandate, then the question has been answered. 

I would note, moreover, that there is some merit in thinking about variation. Take 
the case of the IARP efforts (Cheche Konnen, Funds of Knowledge, etc.). One could 
learn a lot about pedagogy and the organizational features of school which supported 
said pedagogy. On the other hand, if you limit yourself to just comprehensive school 
grants, you would have missed the ability to study these efforts since they were not 
school-level efforts. 

(b) To what extent should the study focus on a before-and-after examination of school 
change as opposed to an examination of the process of change itself? 

Again, this is a methodological issue. The answer to this question should be driven 
by the research question. If you are asking a classical, effects sort of question, then 
you'd want these sort of data, plus some documentation of what's happening in the 
classrooms and schools. If it's a different sort of evaluation, then you might not care 
about these sort of data and, instead, collect samples of student work for scoring and 
analyzing using some sort of rubric. 

(c) To what extent should the study attempt to attribute school change to specific 
federal, state, or local initiatives or programs? 

Given the Department of Education's experiences with the Ramirez study, where 
there was a careful effort to document "treatments" and to find comparable schools, 
but where, due to technical issues, the attribution of cause could not be made (partly, 
I believe, because ED did not like the answer it would have gotten), I believe that this 
sort of research question is both a political and a methodological dead end. Don't 
try. 

My recommended research questions are: 

1. If a school makes pedagogy (i.e., curriculum, instruction, and assessment) for 
LEP students a central concern, then how does the school translate its 
concerns into action? 



What are the relationships between a school mission which takes the 
education of LEP student seriously and Pedagogy (as defined above) (a) in the 
classroom and (b) across the school? 

2. What is the quality, against some sort of standards for pedagogy (i.e., 
standards for the quality of curriculum, instruction, and assessment tasks), of 
school as it is experienced by that school's LEP students? 

Does this quality diverge, in some important ways, from what is experienced 
by other, non-LEP students? 

How is it similar to what other student experience? 

What adjustments are made, in pedagogy, for a school's LEP students? 

3. How do the organizational features of the school (e.g., its shared values and 
structures) support and /or impede efforts to provide students with high 
quality pedagogy? Organizational features include the nature of teachers' 
professional lives, the school's governance systems, the existence of a 
community for teachers and students which includes attention to the 
intellectual substance of what students experience, and the existence of 
accountability systems that provide the school with information about whether 
it is achieving its goals. 

4. How do external agents and agencies-the community, parents, district, state, 
federal, professional organizations, educational reform networks, social sendee 
agencies, etc.-influence the school's efforts to provide high quality pedagogy 
to its LEP students? 

5. What are the complex inter-relationships among the above features, including 
pedagogy, and authentic forms of student learning and /or achievement? 

6. How do the above sorts of things change over time? How are those changes 
related to changes in the relationships explored in #5? 

7. What sorts of predictive (as opposed to purely descriptive, covariation-based) 
models can be generated from these data? How can these models be 
validated? 
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Topic 2: Models for Studying the Content of School Change Efforts 
Written Recommendations from: Betty Mace-Matluck 

(a) Should there be standards relating to the content of educational reform against 
which schools should be judged? If so, what are the key elements of those 
standards? 

The content of the educational reform that a school engages in should be documented 
in relation to the school's perceived needs and resources. It may be that a school will 
decide to go for depth rather than breadth .. and thus work on only one aspect of its 
planned reform at a time ... the context of the reform as well as available resources 
should be the primary elements for investigation. 

The SIAC document does a good job identifying major categories of variables relating 
to the content of educational reform (pages 32-37). 

(b) What are the major categories of variables that should be studied relating to the 
content of educational reform? What are the key variables within each of those 
categories? 

In addition to the variables listed in categories 1-5 on pages 35-36 of SIAC document, 
I would recommend that the following variables be included in the OBEMLA 
Benchmark study: 

1. Structure and characteristics the school— resources available — how these 
are allocated in terms of relevant professional development for all the teachers 
and administrators for upgrading skills and knowledge relative to working 
with LEP students. 

2. Climate and culture — the extent to which and how the school demonstrates 
that it values the home culture(s) and integrates it into the life of the school 
and the extent to which and how the school builds students' self esteem and 
encourages students to hold high expectations for themselves. 

3. Management /Governance — the extent to which and low language minority 
staff are involved in the decision-making process of the school. 

4. Linkages with parents/community — the extent to which and how the 
school involves language minority parents and community 
members /organizations in significant ways in the life of the school — and to 
what extent and how the school ensures that LEP students and family 
members receive needed health and human services (integrated services). 
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5. Student instructional experience — the extent to which and how the school 
meets the affective needs of LEP students, as well as academic needs (i.e., high 
standards and academic programs provided for mainstream students). 

(c) What relationships among categories of content variables or among individual 
variables should receive particular attention in the study? What are the theoretical 
models which suggest the importance of such relationships? 

To what extent and how does the school ensure /develop sensitivity to and concern 
for language and cultural diversity among the school staff. How does this sensitivity 
manifest itself? 
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Topic 2: Models for Studying the Content of School Change Efforts 
Written Recommenadations from: Brian Rowan 

(a) Should there be standards relating to the content of educational reform against 
which schools should be judged? If so, wh.-t are the key elements of those 
standards? 

(b) What are the major categories of variables that should be studied relating to the 
content of educational reform? What are the key variables within each of those 
categories? 

The study should focus on the instruction received by LEP students. Data collection 
should focus on: 

1. How instruction is compatible with culture of learner 

2. Instructional strategy used by teachers of student 

- ambitious content 

- participatory/ constructionist teachers 

3. Coherence at instructional content/strategy across program settings 

4. Student success in instructional assignments /tasks /activities 

(c) What relationships among categories of content variables or among individual 
variables should receive particular attention in the study? What are the theoretical 
models which suggest the importance of such relationships? 

These instructional variables are probably affected by 

1. Program design 

2. Instructional organization/management 

3. School norms/climate 

4. District management context 

5. Community setting 

6. State\ federal trends 
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Topic 2: Models for Studying the Content of School Change Efforts 
Written Recommendations from: Christian Faltis 

(a) Should there be standards relating to the content of educational reform against 
which schools should be judged? If so, what are the key elements of those 
standards? 

I don't feel that there should be standards against which schools should be judged. 
However, in the analysis, the researchers could analyze the data and compare it with 
certain criteria that are assumed to be applied to mainstream students and schools. 
A more informative study might include criteria such as school /classroom climate; 
support services, such as library, sports, assemblies, musical performances, plays, etc.; 
coherence and consistency of coursework within the school, from elementary to 
secondary; coverage of core content material; collaboration among administration, 
teaching, and support staff; and student outcome measures. 

(b) What are the major categories of variables that should be studied relating to the 
content of educational reform? What are the key variables within each of those 
categories? 

I have already mentioned some of the categories in answer (a) above. I would also 
include, however, philosophical stance of administration, faculty and staff about the 
way students learn, about the best ways of teach students from diverse language and 
ethnic backgrounds; community contexts, including external partners, state and 
federal policies; approaches to first and second language /literacy acquisition. Need 
to examine the schools beliefs about ethnicity and language variation. 

(c) What relationships among categories of content variables or among individual 
variables should receive particular attention in the study? What are the theoretical 
models which suggest the importance of such relationships? 

I am not sure how to answer this question, but I feel strongly that the philosophical 
stance of the administrators, principal, teachers and staff need to be connected to 
what happens in the classroom, on the school grounds, and in the community. I also 
think that program variables having to do with L1/L2 acquisition have to be tied to 
classroom strategies. 
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Topic 2: Models for Studying the Content of School Change Efforts 
Written Recommendations from: Charlene Rivera 

(a) Should there be standards relating to the content of educational reform against 
which schools should be judged? If so, what are the key elements of those 
standards? 

I believe we agreed to talk about criteria rather than standards and so have 
responded to the question to address criteria. 

Sample criteria for examining the reform against which schools should be judged 
should include consideration of: 1) what the status quo is within the school context; 
i.e., what courses are being offered to all students? Are the same courses being 
offered to LEP students? Are they of the same quality? How are they adapted to 
address students with different educational and linguistic backgrounds (e.g.. literate 
vs. illiterate etc.); what are expectations for "all" students, including LEP students? 

2) state and local standards as articulated by the school community; and 

3) national standards as appropriate. 

(b) What are the major categories of variables that should be studied relating to the 
content of educational reform? What are the key variables within each of those 
categories? 

Major variables that can be studied are described in the literature review; the models 
provide options for describing key variables and can be drawn from in designing 
the framework for the study. Priorities should include consideration of: a) structure 
and characteristics of the school, b) student instructional experiences; c) 
climate/culture beliefs d) linkages with community/parents, d) 
management/governance. The questions that need to be addressed include: How do 
these change over time? What stimulates change? Who participates in making the 
decisions that create change? How do others who did not participate in the initial 
phase of the change process become convinced to try different strategies /approaches? 
What are the outcomes for students, i.e., what are the indicators that demonstrate 
student growth in academics, English language skills, etc. over specific time frames 
(to be predetermined by the researchers)? 

(c) What relationships among categories of content variables or among individual 
variables should receive particular attention in the study? What are the theoretical 
models which suggest the importance of such relationships? 
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Topic 2: Models for Studying the Content of School Change Efforts 



Written Recommendations from: Diane August 



(a) Should there be standards relating to the content of educational reform against 
which schools should be judged? If so, what are the key elements of those 
standards? 

Since this is an evaluation and not a descriptive study, there should be standards 
which I define as desirable activities or outcomes against which the existing 
conditions are assessed. However, for any given arena (content of reform), the 
standards may differ depending on the assumptions and values of those that 
establish them. This will make establishing benchmarks a formidable task. 
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(b) What are the major categories of variables that should be studied relating to the 
content of educational reform? What are the key variables within each of those 
categories? 

I think we should place priority on examining activities defined to a large extent by 
the legislation. The purpose of the programs are to "reform, restructure, and upgrade 
all relevant programs and operations within an individual school that serve all (or 
virtually all) children and youth of limited-English proficiency in schools with 
significant concentrations of such children and youth. (Acceptable activities are 
further defined in the Act.) Schools will be evaluated for: (1) how students are 
achieving the State student performance standards, if any, including data comparing 
children and youth of limited English proficiency with non-limited English proficient 
children and youth with regard to school retention, academic achievement, and gains 
in English (and where applicable, native language proficiency); (2) program 
implementation indicators that provide information for informing and improving 
program management and effectiveness, including data on appropriateness of 
curriculum in relationship to grade and course requirements, appropriateness of 
program management, appropriateness of the program's staff professional 
development, and appropriateness of the language of instruction; (3) program context 
indicators, and such other information as the Secretary shall require. 

Examples of benchmarks in these arenas include: whether there is a coherent 
program in which federal, state, and local funds are coordinated to enable LEP 
students to meet high standards; whether LEP students have access to curriculum 
aligned with rigorous professional and community standards; whether LEP students 
have coursework that will enable them to graduate om high school with skills 
needed for gainful employment or college; whc.ner there is school level 
accountability connected to performance standards; whether there are mechanisms 
in place for ensuring continuous improvement, etc. 
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(c) What relationships among categories of content variable i or among individual 
variables should receive particular attention in the study? What are the theoretical 
models which suggest the importance of such relationships? 

I can't presume to deal with relationships between categories of content variables 
until we have defined which content variables we will be examining. One model that 
is worth considering is systemic reform since it has been used to frame much of the 
recent Federal legislation. 
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Topic 2: Models for Studying the Content of School Change Efforts 
Written Recommendations from: Julia Smith 



(a) Should there be standards relating to the content of educational reform against 
which schools should be judged? If so, what are the key elements of those 
standards? 

"Should there be" begs the question of "Can there be." The easy answer here is of 
course no - there cannot be a single set of standards against which all educational 
experiences are comparable. However, if you leave the word standard - implying 
equal across context - and try the word characteristic - implying defining features 
which can be recognized and measured, then the question becomes, "Does the content 
of educational reform have identifiable characteristics?" yes, it does. It has people, 
activities, and contexts. 

People: Students, Teachers, Administrators, Parents 

Activities: Language instruction, program coordination, decision making, resource 
allocation, communication between people 

Contexts: Classrooms, Schools, Communities 



(b) What are the major categories of variables that should be studied relating to the 
content of educational reform? What are the key variables within each of those 
categories? 

Each of the above (people, activities, contexts) is a major category of variable which 
should be studied. Points of variation include: 

Students - Background (language, culture, SES, gender, family context) 

Measured behaviors (performance on assessments, entry & exit from programs, 
misbehavior, pro-social or pro-academic behavior) 

Measured beliefs /attitudes (toward school, toward self, toward others) 

Teachers - Background (history in teaching, culture, gender) 

Measured behaviors (mode of delivery for teaching, content delivered to which 
students, involvement in planning & assessment) 

Measured beliefs/attitudes (toward language acquisition, toward students, toward 
school process) 
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Classrooms — Observed structure (# students, time spent on activity) 

Social structure (beliefs and values communicated by teacher, between students) 

School - Observed structure (# students, time spent planning, coordination 
mechanisms) 

Social structure (relationships between teachers, beliefs held about students, beliefs 
held about learning) 

What relationships among categories of content variables or among individual 
variables should receive particular attention in the study? What are the theoretical 
models which suggest the importance of such relationships? 



I 



I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 

ERIC 



Topic 2: Models for Studying the Content of School Change Efforts 
Written Recommendations from: Judith Torres 



(a) Should there be standards relating to the content of educational reform against 
which schools should be judged? If so, what are the key elements of those 
standards? 

If we think of standards more as criteria for evaluating the content of instruction, 
then the study should use current work on academic standards to provide an overall 
frame for evaluating the content of instruction. The research base on general school 
change and that on effective instruction for LEP students may be used to frame the 
major questions to be encompassed by the study, but the study must consider the 
possibility that unanticipated changes may occur, or that the processes of school 
change differ from those described in the research. 

The discussion of Day 2 of the focus group suggests that this study might be used 
to help generate new ways of describing and thinking about the reform/ restructuring 
process, perhaps more as a circular or self-reflective process rather than a linear one. 

A key criterion for indicating the success of the change process should be evidence 
for high academic expectations and the provision of ambitious academic content for 
LEP students in a number of content areas. Criteria for indicating the success of 
organizational change could be drawn from increased levels of communication, 
evidence of joint planning, and observations of changes in teacher practice over time. 



(b) What are the major categories of variables that should be studied relating to the 
content of educational reform? What are the key variables within each of those 
categories? 

Beyond seeking evidence of a shared vision/high expectations and ambitious 
academic content (in its varied subcategories), the study should seek evidence for 
changed relationships and patterns of communication among teachers, teachers and 
parents, and teachers and school administrators. I would want to investigate how 
Fede^.l and other resources have been used to leverage or facilitate these changes. 
I would want to know what resources have been used to support changes in 
curriculum and instruction over time, and to what results in classrooms. The 
literature review indicates a number of additional areas for exploration. 



What relationships among categories of content variables or among individual 
variables should receive particular attention in the study? What are the theoretical 
models which suggest the importance of such relationships? 

Clearly, the study should examine the relationships between configurations of 
resources and specific change efforts (if any may be observed). Context variables will 
be needed to frame the research, because we expect that the focus and content of 
change will vary in response to these (for example, district, state, Federal policy; 
cultural and linguistic backgrounds of student groups in the school; SES of these 
groups and associated attitudes toward school in general and instruction in 
particular). Also important would be the educational strengths and experiences of 
the students to be served - first-language literacy and years of education completed 
outside the US, for example. 

Teacher participation in curriculum planning and school governance may also be 
critical in supporting change. The study could look for relationships between 
communication among teachers regarding effective practices, teachers' changing 
expectations for student learning, teachers' changing involvement in curriculum 
design, and student engagement in learning. The resulting changes in teachers' 
beliefs and knowledge may result in changes in teacher behaviors in classrooms. 
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Topic 2: Models for Studying the Content of School Change Efforts 
Written Recommendations from: Paul Berman 



(a) Should there be standards relating to the content of educational reform against 
which schools should be judged? If so, what are the key elements of those 
standards? 

The word "standards" is inappropriate as used here. I think criteria is more 
appropriate. Yes, either criteria should be suggested by the proposer or, better yet, 
the proposer should specify how criteria would be developed by the study team. In 
addition to criteria concerning the content of education reform, criteria should be 
specified about student performance; such criteria may include benchmarking to 
national, state or local academic or skill standards. 

(b) What are the major categories of variables that should be studied relating to the 
content of educational reform? What are the key variables within each of those 
categories? 

Major categories are changes in: student performance and/or learning for LEP 
students, LEP students, and other students; instruction and curriculum; the language 
acquisition strategies; the coherence and coordination of curriculum and instruction 
over time and between teachers and developmental levels; the school organization, 
culture and structure and linkages to parent/community and others outside the 
school. The proposer should specify a beginning list of variables within categories 
(which constitute hypothesis based on the literature. 

(c) What relationships among categories of content variables or among individual 
variables should receive particular attention in the study? What are the theoretical 
models which suggest the importance of such relationships? 

The answer depends on resources for research. The key is to relate comprehensive 
institutional change to changes in teacher practices as expressed in curriculum and 
instruction and both of these to changes (hopefully growth) in student performance. 
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Topic 2: Models for Studying the Content of School Change Efforts 
Written Recommendations from: Walter Secada 



(a) Should there be standards relating to the content of educational reform against 
which schools should be judged? If so, what are the key elements of those 
standards? 

Yes. The school's pedagogical program should be judged based on standards for: 

1. Ambitious forms of pedagogy (i.e., curriculum, instruction, and assessment); 

2. The match between what is provided to LEP students and what is provided 
to non-LEP students; 

3. Coherence across curriculum, instruction, and assessment; curriculum 
coherence across grades; 

4. Connections in pedagogy within the various disciplinary content areas; 
connections across school subjects; and finally, connections between what 
students experience in school and the world they actually live in outside of 
the school (also known as value beyond the class). 

Student achievement should also be judged against standards for authentic student 
learning. 

(b) What are the major categories of variables that should be studied relating to the 
content of educational reform? What are the key variables within each of those 
categories? 

I would have seven outcome variables: 

1. Authentic student achievement; 

2. Ambitious pedagogy; 

3. Equity; 

4. Existence of a community for both teachers and students which includes a 
focus on student learning; 

5. Forms of school-level empowerment for teachers and students (school 
governance, the ability to exert leadership and influence, etc.); 
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6. Relationships between the school and external, community-based resources 
and agencies for providing comprehensive services to students; 

7. The existence of accountability systems through which the school would 
receive information about the efficacy of its efforts. 

I have tried to clump the above in terms of the relative importance. The reason I 
consider each of them as outcomes is that, were a school to change on the any 1 of 
these (but not the others, which seems unlikely, I know), I would be willing to say 
that the school is a better place for LEP students than if it had not changed on that 
1 item. 

I am not sure that I can specify the independent variables as easily as the dependent, 
since my question would be: what in the school supported the creation of any 
combination of the dependent variables. I would then call those things which 
supported (and /or interfered with) the school's outcomes and WHICH COULD BE 
MANIPULATED through some sorts of policy initiatives, independent variables. 
Those things which supported, but could not be manipulated, I would label, 
covariates. 

An important thing, which cuts across all categories of variables involves two things: 

1. Is there some sort of congruence between the cultures which the students 
bring with them from home and what they experience in the school and in the 
classroom? 

2. How do schools and classroom facilitate students' being able to cross cultural 
borders? 



What relationships among categories of content variables or among individual 
variables should receive particular attention in the study? What are the theoretical 
models which suggest the importance of such relationships? 

See above. Also, I think that three things would cut across variables: 

1. Concern for the intellectual quality of what students experience OVER concern 
for what to do as an end in itself. For instance, cooperative groups are just 
a means to an end (student learning of ambitious content); they are not the 
end in and of themselves. 

2. Sustained effort OVER short term, one-shot efforts which are constantly 
changing. 

3. Creation and maintenance of communal forms of organization for students 
and teachers which include (a) concern for 1, above and (b) elasticity to 
accommodate reasonable forms of diversity OVER rigid, bureaucratic forms 
of organization. 
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I Also, I would look at how these things vary as functions of social class and other 

larger contexts. 
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Topic 3: Models for Studying the Process of School Change Efforts 
Written Recommendations from: Betty Mace-Matluck 

(a) Should there be standards relating to the process of school change against which 
schools should be judged? If so, what are the key elements of those standards? 

There are theoretical models of the change process (based on studying examples of 
change) that can inform the study— There are predictable stages and concerns that 
can be expected as school staff move through the change process — the study should 
document these at the beginning of the study for each school in the sample and 
document and examine the progress and factors influencing the progress over time. 
I agree that change is not linear and that context variables also change and thus the 
"content of the education reform" may be "revised" appropriately before 
"institutionalization" because of change requiring a shift in content of educational 
reform before a given change effort has become institutionalized. 

(b) What are the major categories of variables that should be studied relating to the 
process of school change, including factors which promote or impede change? 
What are the key variables within each of those categories? 

Recognition of need to change — data to support the kind of needs around which 
change can be planned and implemented; the development of a shared vision for the 
school, developed by broad representation from the stakeholders, evolutionary 
planning, resulting in action plans with benchmarks; support mechanism 
(professional development and resources assistance); monitoring and problem solving 
along the way; formal and self-evaluation of accomplishments and process; recycling 
of the process to appropriate steps as needs and or context changes. 

(c) What relationships among categories of change variables or among individual 
variables should receive particular attention in the study? What are the theoretical 
models which suggest the importance of such relationships? 

Representation of minority community in all phases — particularly in vision building 
and planning stages. Involvement of external change agents to bring new and 
relevant knowledge . and expertise to the process — and to provide constructive 
criticism. 
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Topic 3: Models for Studying the Process of School Change Efforts 
Written Recommenadations from: Brian Rowan 

(a) Should there be standards relating to the process of school change against which 
schools should be judged? If so, what are the key elements of those standards? 

I agree with the general thrust of the afternoon session: We need to discover how 
change is produced NOT specify it in advance. The only proviso 1 would add is that 
we should examine the changes required /involved at three levels 

1 . organizational change processes 

2. teacher/other staff change process 

3. student /change processes 

(b) What are the major categories of variables that should be studied relating to the 
process of school change, including factors which promote or impede change? 
What are the key variables within each of those categories? 

I could make up a list here, but given the state of theory in this field, I'm not sure 
it would be helpful. Here it is (in no particular order): 

political dynamics, timing, stressors, decisionmaking, visioning (culture-building), 
leadership, control processes, planning, information processing, monitor, adjust, 
evaluation. 

(c) What relationships among categories of change variables or among individual 
variables should receive particular attention in the study? What are the theoretical 
models which suggest the importance of such relationships? 
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Topic 3: Models for Studying the Process of School Change Efforts 
Written Recommendations from: Christian Faltis 

(a) Should there be standards relating to the process of school change against which 
schools should be judged? If so, what are the key elements of those standards? 

I would say no. I would like to see that any standards or criteria relating to the 
process of change stem from a thorough analysis of and grounded in the data. The 
basic question here, for me, is "When a school decides to make a schoolwide change 
to include LEP students in the all aspects of schooling, what are the kinds of things 
that they have to consider once they begin the change?" How does the philosophical 
stance evolve and how is it acquired by the faculty and staff who didn't share it 
initially? What are the kinds of things (discourse, knowledge, strategies, etc.) that 
participants in the reform process learn? 

(b) What are the major categories of variables that should be studied relating to the 
process of school change, including factors which promote or impede change? 
What are the key variables within each of those categories? 

I guess I would say that in addition to the content variables listed above, I would 
also include philosophical stance; ways of handling resistance; quality of instruction; 
and knowledge base. 

(c) What relationships among categories of change variables or among individual 
variables should receive particular attention in the study? What are the theoretical 
models which suggest the importance of such relationships? 

Again, I would pay attention to how the philosophical stance changes shape over 
time, and see how it related to other changes occur over time. 



Topic 3: Models for Studying the Process of School Change Efforts 
Written Recommendations from: Charlene Rivera 



(a) Should there be standards relating to the process of school change against which 
schools should be judged? If so, what are the key elements of those standards? 

The standards should be the same that are being used to judge the implementation 
of GOALS 2000. This suggests a need to make sure that all benchmark studies 
funded through the Department of Education are compatible and that they are 
working to measure the same standards. If different standards are set for different 
studies, this benchmark study may become a useless exercise that will contribute 
little knowledge to the main question: How are LEP students being treated /faring 
in the reform effort? Are the reforms implemented as part of GOALS 2000 and 
funded through Title VII contributing to a significant change in how LEP students 
are treated in schools? Are the changes contributing to a belief that all faculty are 
responsible for LEP students', well-being, achievement, etc? 

(b) What are the major categories of variables that should be studied relating to the 
process of school change, including factors which promote or impede change? 
What are the key variables within each of those categories? 

Variables to be studied should include consideration of how the system is changing 
to integrate LEP students so they are not marginalized and, thus, treated differently 
than other students. Critical variables to be studied should include context, 
instruction, school structural variables (e.g. how does a faculty develop a mission/ 
shared vision that motivates mainstream and LEP teachers to collaborate? How do 
directives from school and /or district level leaders get communicated to a faculty? 
What structures facilitate/impede communication of information? Are there patterns 
of communication that exclude certain groups or types of teachers? What are the 
implications of a communication style within a school for implementing changes that 
are effective for all students, including LEP students? How do the educators within 
the school keep themselves and the school renewed? What are the support 
mechanisms that enable substantive and positive change to t ike place? What inhibits 
these?) 

(c) What relationships among categories of change variables or among individual 
variables should receive particular attention in the study? What are the theoretical 
models which suggest the importance of such relationships? 
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Topic 3: Models for Studying the Process of School Change Efforts 
Written Recommendations from: Diane August 



(a) Should there be standards relating to the process of school change against which 
schools should be judged? If so, what are the key elements of those standards? 

I still really have trouble understanding the distinction between the process of school 
change and the content of school change, unless you are referring to the content of 
school change as goals and objectives and the process as how these goals and 
objectives are accomplished or stymied. If you mean it as the latter, what we are 
talking about are methodologies to use to examine whether something has been 
implemented, how well, and why. For example, I would define institutional learning 
as the content of school change (that is, as something that has taken place or not) and 
the process as what happened to make it take place or not. Because I'm not sure 
what is meant by the process of school change, I can't elaborate further. 

(b) What are the major categories of variables that should be studied relating to the 
process of school change, including factors which promote or impede change? 
What are the key variables within each of those categories? 



(c) What relationships among categories of change variables or among individual 
variables should receive particular attention in the study? What are the theoretical 
models which suggest the importance of such relationships? 
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Topic 3: Models for Studying the Process of School Change Efforts 
Written Recommendations from: Judith Torres 



(a) Should there be standards relating to the process of school change against which 
schools should be judged? If so, what are the key elements of those standards? 

The discussion on Day 2 suggested that the literature on school change may not 
necessarily be most useful in framing parameters within which to describe the change 
process in schools. I have come to the end of the discussion process thinking that 
this study might be most useful if it seeks to expand or generate stronger, more 
sensitive representations of school reform as a self-reflexive process, or as a series of 
problem-solving adventures. It also seems likely that school reform actually is a 
sequence of different kinds of problem-solving efforts, because the challenges to 
reform efforts are different at various points in the restructuring process. This 
approach also recognizes that problem-solving is not necessarily successful at all 
points, and thus might lead the researchers to think of ways that the process might 
be "unstuck" or refocussed. 

Although this is not directly relevant here, it will be vital not to lose sight of the 
actual areas in which change was attempted (or not attempted) in the selected 
schools. An analytic framework must be sufficiently flexible to accommodate to the 
variety of school experiences. 



(b) What are the major categories of variables that should be studied relating to the 
process of school change, including factors which promote or impede change? 
What are the key variables within each of those categories? 

Process variables: resources, participants, activities, communication, time. Again, I 
would want to see an examination of. what resources were used to facilitate the 
change process, and how they were used. These might include time, funds for 
training, materials, travel, coursework, classroom coverage, and computer hardware 
and software. The study should examine who was involved in the change process, 
how extensive their involvement was, and how it was undertaken. What participants 
actually did, and with whom, will be important as well. What technical assistance 
was available is also important. (See the literature review on pages 66 and 67.) 

Students are part of this process. Identifying how their experiences changed as a 
result of other systemic changes is essential. The role played by school leadership 
will also need to be described carefully, as will changes in the active involvement of 
parents and other community members in the school's educational processes. 
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What relationships among categories of change variables or among individual 
variables should receive particular attention in the study? What are the theoretical 
models which suggest the importance of such relationships? 

It would be important to determine whether or not school initiatives and funding 
sources are worked into a coherent whole; that goals and processes are 
complementary and mutually supportive. It may also be important to examine the 
content of change as carried out through discussion, deliberation, and planning at the 
school level. 




Topic 3: Models for Studying the Process of School Change Efforts 
Written Recommendations from: Walter Secada 



(a) Should there be standards relating to the process of school change against which 
schools should be judged? If so/ what are the key elements of those standards? 

I would ask three questions: 

1. Is the re ambitious and coherent pedagogy? What supports that pedagogy in 
the school? 

2. How did that pedagogy come about? 

3. How did those supports come about? 

In other words, I would focus on student experiences (pedagogy and achievement) 
and go backwards as to how the things which support it came about. 

(b) What are the major categories of variables that should be studied relating to the 
process of school change, including factors which promote or impede change? 
What are the key variables within each of those categories? 

Again. 



(c) 



What relationships among categories of change variables or among individual 
variables should receive particular attention in the study? What are the theoretical 
models which suggest the importance of such relationships? 



Topic 4: Methodological Approaches to Studying School Change 
Written Recommendations from: Betty Mace-Matluck 

(a) Should the study employ a broad mix of methodological approaches, or should it 
focus on the strong application of one or two approaches? 

Four levels of study (student, classroom, school, context variables), lesser focus on 
student outcomes if resources are a factor. Probably will need a mix of 
approaches— but let the research questions drive the methodology. 

(b) What specific approaches should be employed? If a mix of approaches is 
suggested, how much of the study's resources should be devoted to each approach? 

Case study, with qualitative data, as well as some quantitative data collected across 
sites— that may allow the development of insight into the case studies 

(c) What specific approaches are recommended for collecting data over time on 
teaching/learning environments and teacher/learner relationships? 

(d) What are the design requirements of the methodologies which are recommended 
(i.e., how many measurements of what type with whom and over what time period 
are required)? 

(e) What is the recommended role of investigators in the change research in terms of 
objectivity and involvement in the change process? Should they make 
recommendations and/or be involved in implementing changes? 

Researchers should provided feedback to sites regarding instruments and data— and 
should engage in dialogue with the site practitioners regarding their view about what 
the data may mean (i.e. assist in interpreting the results). However, I strongly feel 
the researchers should not serve as "the external agent". I doubt that they would 
have the expertise and skills needed to do this— and furthermore, this will affect the 
outcome of the study at different sites— given the ability and resources of different 
research team members. 



Topic 4: Methodological Approaches to Studying School Change 
Written Recommendations from: Brian Rowan 

(a) Should the study employ a broad mix of methodological approaches, or should it 
focus on the strong application of one or two approaches? 

I'd recommend a mix, probably, in the context of a multi-site 

(b) What specific approaches should be employed? If a mix of approaches is 
suggested, how much of the study's resources should be devoted to each approach? 

Case study with standard questions /issues studied at each site. Data should be 
collected at multiple levels a) students b) teacher c) school-organization 
d) district /community 

(c) What specific approaches are recommended for collecting data over time on 
teaching/learning environments and teacher/learner relationships? 

For student data, look at instructional tasks worked on, success, instructional 
strategies used, cultural congruence of instruction. Also, engagement in school- 
particularly, total school environment. 



(d) What arr the design requirements of the methodologies which are recommended 
(i.e., how many measurements of what type with whom and over what time period 
are required)? 

My main recommendation is a multisite case study where similar information is 
collected over time at all research sites. The mix of methods /measures to be 
determined by research questions in RFP. 



(e) What is the recommended role of investigators in the change research in terms of 
objectivity and involvement in the change process? Should they make 
recommendations and/or be involved in implementing changes? 

4 possible soles 

HI < levels of research involvement > lO 

Research directed < change process > practitioner-di ?cted 



- Planned variation model. Developers have explicit model of education for LEP 
students, they implement at multiple sites, and evaluate it using standard methods. 



- Organizational development external change agent supports development of 
school-initiated models and studies process 

- Evaluation /feedback model researcheis feedback result, provide info to sites, hold 
survey feedback sessions, etc. 

- Non-involvement researchers gather data but do not share. 



Topic 4: Methodological Approaches to Studying School Change 
Written Recommendations from: Christian Faltis 

(a) Should the study employ a broad mix of methodological approaches, or should it 
focus on the strong application of one or two approaches? 

4 

If by methodological approaches, you mean qualitative and quantitative, I would say 
that you shbuld use whatever you need within each approach to answer the research 
questions you have posed. 

(b) What specific approaches should be employed? If a mix of approaches is 
suggested, how much of the study's resources should be devoted to each approach? 

I think that in order to study the process of change that researchers will have to 
employ qualitative approaches, including observation, interviews, and perhaps 
videotaping. These will need to be done in multiple contexts with many different 
consultants and participants at various points in time. The study should also collect 
data on student discourse, student tasks, views on the classroom, the teacher, school 
etc. There can also be lots of other kinds of quantifiable data collected. It depends 
on the research questions. The amount of resources devoted to each type depends 
on the foci of the study. To get at institutional change at a deep level requires 
intensive qualitative study. However, much can be learned from surveys and 
assessment data. Moreover, it is important to have assessment data in order to learn 
something about how well LEP students are doing compared to other students, and 
how well school is doing compared to some national criteria. However, I don't feel 
that this should be the primary focus of the study. 

(c) What specific approaches are recommended for collecting data over time on 
teaching/learning environments and teacher/learner relationships? 

depends on your research questions, but if most Hkely, lots of classroom and other 
significant school action environments using fieldnotes, or a system of observation 
that looks at amount and types of participation and interaction, followed by 
interviews with teacher, librarian etc. to get at what is going, why, what is the 
purpose, how are students learning, etc. Could use video-tapes of lessons, and then 
do stimulated recall sessions. 

To get a teacher-student relationships, you again need to do lots of observation and 
interviews of both the students and teacher. With teacher, you want to learn how 
the teacher interacts with students, how many times students are called up, what 
kinds of questions are asked with different students, movement patterns in the 
classroom, what the teacher knows about each student, about the student's family, 
community, and ethnic background and history (without access to notes). Get at 
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students' perspective of the teacher, the classroom and the school, and to other 
students. Are there teacher's pets, who are the stars of the class, do students feel 
they are participating equally in the class and school activities? 



(d) What are the design requirements of the methodologies which are recommended 
(i.e., how many measurements of what type with whom and over what time period 
are required)? 

this is a tough one to answer. For certain kinds of information-, it is necessary to 
spend lots of time initially to figure out what is going on in the school, and for other 
types of information, it is important to take a reading at least once a year. It also 
depends on the cost of the data collection. For example, since classroom observation 
is very expensive, you might want to do on a relatively few occassions in the first 
couple of years of the study when money is less available, and then become more 
intense in the 3rd and 4th year. I would think, however, that minimally, researchers 
should take end of the year data on grades, portfolios and other assessment products, 
and should also be interviewing and observing in sites throughout the year. 

(e) What is the recommended role of investigators in the change research in terms of 
objectivity and involvement in the change process? Should they make 
recommendations and/or be involved in implementing changes? 

I think that it is unavoidable that researchers will not be involved at least indirectly 
in the change process. They are talking with many people, some of whom are very 
much involved in the reform, and these people will learn from these discussions. I 
think that when researchers have discussions and /or debriefing sessions with 
teachers /librarians etc., that embedded in the discourse are suggestions and ideas 
that eventually may find there way into the reform process. I don't think that 
researchers should be directly involved in planning changes or in telling school 
people what to do. They may offer suggestions or ideas. 
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Topic 4: Methodological Approaches to Studying School Change 
Written Recommendations from: Charlene Rivera 

(a) Should the study employ a broad mix of methodological approaches, or should it 
focus on the strong application of one or two approaches? 

A mix of approaches will probably be needed to carry out the study and these will 
need to be defined in the proposal; however, within the first two years of the study, 
as the contractor learns more about the cases under study and has further refined the 
research questions to be addressed, there should be flexibility to modify and/or add 
approaches, as appropriate and needed. However, these modification in the this 
phase of the study should not be so sharp as to make the data collected non 
compatible with data to be collected in the future. 



(b) What specific approaches should be employed? If a mix of approaches is 
suggested, how much of the study's resources should be devoted to each approach? 

In general, the approach to the study should be driven by the research questions; 
these should be developed with the goal of understanding the instructional context, 
including the role of school leadership; how the school is changing with the 
implementation of GOALS 2000 and the funding of Title VII comprehensive school 
grants, how the changes stimulated by these reforms are making a difference in the 
instructional climate, mainstream and LEP teacher expectations, supports for teachers 
to understand how to provide comprehensible instruction to LEP students that leads 
to high levels of academic growth comparable to that of other students in the school. 

(c) What specific approaches are recommended for collecting data over time on 
teaching/learning environments and teacher/learner relationships? 

Approaches should include: teacher and student interviews, teacher and student 
focus groups, teacher logs, phone interviews with teachers, classroom observations, 
observations of select individual students for whom student work is collected over 
time. Instruments should be standardized across sites; case study personnel should 
be bilingual /multilingual and assigned to a site based on their knowledge of the 
home cultures oi the students in a school; they should be trained to be skilled 
observers. 



(d) What are the design requirements of the methodologies which are recommended 
(i.e., how many measurements of what type with whom and over what time period 
are required)? 



What is the recommended role of investigators in the change research in terms of 
objectivity and involvement in the change process? Should they make 
recommendations and/or be involved in implementing changes? 



The investigators will need to gain the confidence of individuals who will be sampled 
over time within a school community. This will require providing assurances that 
regular instructional processes will not be impeded by the presence of researchers 
and that resources will be provided, as necessary and appropriate, to enable 
participating educators to complete questionnaires, participate in focus groups etc. 

Moreover, if in-depth insights are to be provided by school personnel over an 
extended period of time, participating educators will need to receive feedback and 
be regularly encouraged by the researchers to document and reflect on the change 
process. (How they are involved in creating, implementing, and sustaining change; 
how LEP students are being treated as a result of the change?) In addition to 
motivating positive participation in the study, feedback to teachers and other 
educators involved in the study will help researchers clarify and validate findings; 
it will also provide opportunities for the educators involved at a school site to extend 
and amplify their reflections on the change process and its effects on LEP students. 

Since resources may be limited, it will probably be necessary to limit the number of 
sites where intensive involvement of educators and students is required. 
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Topic 4: Methodological Approaches to Studying School Change 
Written Recommendations from: Diane August 

(a) Should the study employ a broad mix of methodological approaches, or should it 
focus on the strong application of one or two approaches? 

The study should approach a broad mix of approaches. 

(b) What specific approaches should be employed? If a mix of approaches is 
suggested, how much of the study's resources should be devoted to each approach? 

The approaches will depend on what variables are being assessed. For example, to 
examine whether there is articulation and coordination between the bilingual 
program and the "regular" school program will require the use of questionnaires and 
observations. The amount of resources devoted to each approach will depend on 
how important you think the particular variable under consideration is. 

(c) What specific approaches are recommended for collecting data over time on 
teaching/learning environments and teacher/learner relationships? 

Use of observation instruments and interviews. I recommend looking at Andy 
Porter's work on enacted curriculum and possibly what New York is doing with their 
new inspectorate system. 

(d) What are the design requirements of the methodologies which are recommended 
(i.e., how many measurements of what type with whom and over what time period 
are required)? 

Design requirements will depend on overall research design and independent 
variables. 



(e) What is the recommended role of investigators in the change research in terms of 
objectivity and involvement in the change process? Should they make 
recommendations and/or be involved in implementing changes? 

Investigators should not be involved in the change process while the study is taking 
place. Investigators will have to find ways to establish rapport with school staff in 
order to obtain accurate information. 
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Topic 4: Methodological Approaches to Studying School Change 
Written Recommendations from: Julia Smith 

(a) Should the study employ a broad mix of methodological approaches, or should it 
focus on the strong application of one or two approaches? 

Whichever focus chosen needs to be informed by information from others. For 
example, if taking a strong quantitative analysis survey measures are vastly improved 
through qualitative validation (interview of a random set, observation of practice). 
Conversely, even within strong anthropological approach, there is much to be learned 
by surveying a larger population on topics and themes which emerge. But the 
method should serve the question -some questions are simply not answerable by 
interview, others can only be. 



(b) What specific approaches should be employed? If a mix of approaches is 
suggested, how much of the study's resources should be devoted to each approach? 

See last line of a 

(c) What specific approaches are recommended for collecting data over time on 
teachingAearning environments and teacher/learner relationships? 

Not sure how this can be dealt with outside the proposals solicited 

(d) What are the design requirements of the methodologies which are recommended 
(i.e., how many measurements of what type with whom and over what time period 
are required)? 

A specific - One method, which is appropriate for answering one type of question, 
has some very specific needs -> If one is trying to assess student growth over time 
in different contexts, one needs at least 3 (5 is better) measures from the students. 
RASCH technique can be effectively used to establish a reliable set of responses on 
a few items (20-30). It allows one to look at growth over time on equated measures 
without having to write 5 different versions of similar items or impose a ceiling 
effect. 



i.e. 20 each, each set linked to form a single index of performance. 
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basic < 100 items > advanced 

The cross-level model needs information from (minimum) 20 students in each context you 
want to address. Ideally the design has (min) 10 in each cell where interaction is a concern. 

i.e. consider students number 1-50 in program A, B, C 
X = in the program, O = not in program 
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You use this to set up a matrix of related contexts. 



(e) What is the recommended role of investigators in the change research in terms of 
objectivity and involvement in the change process? Should they make 
recommendations and/or be involved in implementing changes? 
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Topic 4: Methodological Approaches to Studying School Change 
Written Recommendations from: Judith Torres 

(a) Should the study employ a broad mix of methodological approaches, or should it 
focus on the strong application of one or two approaches? 

The methodologies need to be tailored to the research questions, as well as the "level" 
of the study - student, teacher, school, "process," or context. Each implies a range 
of methodologies and different data collection points. Optimally, resources should 
be focussed as close to the educational experiences of the child as possible, describing 
the content of instruction as experienced by the child. This would include classroom 
observations and reviews of student work. It might also include group interviews 
with students. 

The focus on instruction also implies working with teachers, observing teacher 
behaviors, reviewing plans, and discussing behaviors with teachers to the extent that 
resources permit. 

To derive information on change processes, teacher interviews, focus groups, or 
surveys would also be appropriate. Observations of planning meetings might also 
be helpful. Interviews with principals will also be important, as would focus groups 
with parents. 

(b) What specific approaches should be employed? If a mix of approaches is 
suggested, how much of the study's resources should be devoted to each approach? 

Please see (a) for some of this. Deciding what the mix should be depends on the 
available resources and the study questions. 

(c) What specific approaches are recommended for collecting data over time on 
teaching/learning environments and teacher/learner relationships? 

Some of this has been described above. It is best derived at the "micro" level through 
observations and descriptions of behaviors in classrooms. It can also be confirmed 
with interviews with children and parents. 
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What are the design requirements of the methodologies which are recommended 
(i.e., how many measurements of what type with whom and over what time period 
are required)? 

This would require considerable resources to be done well, because the observations 
would have to be fairly intensive. Analysis of qualitative information is also 
expensive. 

What is the recommended role of investigators in the change research in terms of 
objectivity and involvement in the change process? Should they make 
recommendations and/or be involved in implementing changes? 

I cannot see how researchers can be involved with their subjects without something 
being exchanged between them. This will result in the researchers contributing in 
some ways — references, contacts, reflections on practice. It should not be directive, 
but it may be reflective and constructive. It may be formative and ongoing, but this 
should emerge in the process. It may be desirable to include some provisions for 
feedback in the proposal. I do not see their involvement in implementing changes 
as desirable, having been coopted into that role in the past. It creates difficult role 
conflicts. 



Topic 4: Methodological Approaches to Studying School Change 
Written Recommendations from: Walter Secada 

(a) Should the study employ a broad mix of methodological approaches, or should it 
focus on the strong application of one or two approaches? 

(b) What specific approaches should be employed? If a mix of approaches is 
suggested, how much of the study's resources should be devoted to each approach? 

(c) What specific approaches are recommended for collecting data over time on 
teaching/learning environments and teacher/learner relationships? 

(d) What are the design requirements of the methodologies which are recommended 
(i.e., how many measurements of what type with whom and over what time period 
are required)? 

(e) What is the recommended role of investigators in the change research in terms of 
objectivity and involvement in the change process? Should they make 
recommendations and/or be involved in implementing changes? 

(1) Research questions should drive methodology and approaches. 

(2) Recognize that sample drives /limits the sort of methodologies which are 
possible and /or desirable. For instance, many "progressive" schools (which are 
trying to engage in ambitious forms of pedagogy) have an ideological 
aversion to testing. 

(3) Possibility of gathering samples of student work and scoring them against 
generic rubrics should be considered. 

(4) Design should allow for pursuit of site-based issues which should illuminate 
important processes. 

(5) Select based on evidence of ambitious pedagogy for LEP students (i.e. some 
combination of curriculum, instruction, and assessment) 

(6) Recognize that not all subject areas can be equally weighted. Focus on a few. 

(7) Broader environments - than just what happens in the classroom — are 
important. Look at library, cafeteria, etc. 
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Some information - e.g. students' sense of self, or teachers' professional 
culture, or existence of actual teacher planning — need to be gathered through 
a coordination of interviews (small group) and surveys. 

Possible sampling 

A) - select "mature" programs. I.e. where a school has been doing X for 4 or 

5 years already 

- use this small sample to create list of characteristics that support 

pedagogy. 

B) - follow mature schools over rest of study 

- select new schools (year 2 + 3) to add to sample and follow them based 

on characteristics 

The real strength of multi-methodological studies lies in the play between the 
quantitative and the qualitative data. 

a) use the quantitative results to test and validate hypothesis generated from 
the qualitative. 

b) use the qualitative case to illuminate the qualitative results and processes, 

c) use the qualitative results to show the limits of the quantitative results. 
Support schools by 

a) paying for substitute teachers during site visits; paying teachers for their 
time; 

b) paying for each site to conduct an action-research study on its own; 

c) bringing research sites together so that they can share — common problems, 
issues and success stories. 



Topic 5: Methodological Issues in Studying School Change 
Written Recommendations from: Betty Mace-Matluck 

(a) What are the key issues which complicate the design of the study (e.g., sampling, 
number of measurements)? How can the study best address those issues? 

New federal strategy — samples of potential sites are not available until awards are 
granted. Pool of Title VII schools in the early years of the study are likely to be . 
atypical of schools in general that serve LEP students. Sample from non-Title VII 
schools as well. Select some for indepth study, others for less intensive study; add 
a wider range of schools in Year II. 

(b) What are the key issues which threaten the validity of measurement in the study 
(e.g., the level of measurement, appropriateness of measures for LEP students)? 
How can the study best address those issues? 

Special populations focus (LEP students)— appropriate and valid measures that look 
at student academic growth are likely to be labor and resource intensive (e.g. 
performance assessment vs "off-the-shelf standardized measures). Therefore may 
need to focus study resources to a greater extent on changes in student instructional 
experiences as opposed to change in student achievement. 

(c) What are the key issues which will complicate the analyses of the study data (e.g., 
the presence/absence of standards for assessing change, difficulty in drawing 
causal inferences)? How can the study best address those issues? 

Absence of standards for assessing change — this will affect design and possible 
expected outcomes of the study. Caution: Give thought up front to design of database 
for future possible use by potential audiences. 
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Topic 5: Methodological Issues in Studying School Change 
Written Recommendations from: Brian Rowan 



(a) What are the key issues which complicate the design of the study (e.g., sampling, 
number of measurements)? How can the study best address those issues? 

the key issue is funding. More = wider data collection/analysis 

(b) What are the key issues which threaten the validity of measurement in the study 
(e.g., the level of measurement, appropriateness of measures for LEP students)? 
How can the study best address those issues? 

(c) What are the key issues which will complicate the analyses of the study data (e.g., 
the presence/absence of standards for assessing change, difficulty in drawing 
causal inferences)? How can the study best address those issues? 



I do not feel able to address these issues in the abstract. Not knowing money levels, 
research questions, research audiences, I cannot think sensibly about design issues. 



Topic 5: Methodological Issues in Studying School Change 
Written Recommendations from: Christian Faltis 



(a) What are the key issues which complicate the design of the study (e.g., sampling, 
number of measurements)? How can the study best address those issues? 

I think the key issues that complicate this design are that 1) it is a mixture of a policy 
study and a research study; 2) that there are no good standards against which some 
of the student outcome data may be compared; 3) it is not clear whether or not the 
study can produce benchmarks; and 4) the sample is ill-defined. 

(b) What are the key issues which threaten the validity of measurement in the study 
(e.g., the level of measurement, appropriateness of measures for LEP students)? 
How can the study best address those issues? 

A reliance on surveys, test scores, checklist systematic observation, and relying 
exclusively on second-hand information about students teachers, and others. Another 
potential issue is that the funding source would include too high a number of schools 
so that it would be difficult to make any sense of the contextual variables present at 
the individual school sites. 

The best way to address these issues is to insure that researchers employ highly 
trained (language appropriate) research teams that know how to conduct qualitative 
research and spend lots of quality time in the schools. 

(c) What are the key issues which will complicate the analyses of the study data (e.g., 
the presence/absence of standards for assessing change,, difficulty in drawing 
causal inferences)? How can the study best address those issues? 

Qualitative data is always difficult to analyse and very time consuming. It is 
important that the researchers state clearly as possible the method of analysis to be 
used. Also it will be difficult to fit qualitative data into quanitative data. It can be 
done in lots of ways, but is normally very difficult to do well. Researchers should 
state how they plan to interweave and make sense of the two types of data analyses. 

It wih probably be difficult to make inferences about the specific impact of Title VII 
on school change, but this is inherent in this kind of study. However, the researchers 
should be able to say lots about what is happening in the school since the time that 
the school receives the grant and make good guesses about the value Title VII has 
added. 1 

I think that if researchers go into the school and focus on institutional changes in the 
conditions that enable LEP student greater access to, participation in, and benefit 
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■ from classroom and school activities, they will be able to come away with some good 

ideas about what kinds of things happened, and perhaps need to happen so that LEP 
£ students are included equally in all aspects of schooling. 
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Topic 5: Methodological Issues in Studying School Change 
Written Recommendations from: Charlene Rivera 



(a) What are the key issues which complicate the design joi the study (e.g., sampling, 
number of measurements)? How can the study best address those issues? 

Key issues which complicate the design of the study include: a) design of data 
collection instruments, b) identification of appropriate student measures; c) criteria 
" for sampling schools; d) criteria for selecting some schools to be studied in intensive 
ways and others to be studied less intensively. 

To best address: 

a) use already developed instruments, such as NELS questionnaires, if available; b) 
consider use of assessment processes already in place to assess students within a 
school; if measures appropriate for LEP students are not in use, alternative 
performance measures may need to be identified. Limit the number of content areas 
to be assessed and performances /scores to be reported; c) include as many schools 
with Title VII comprehensive school programs in the study as the resources allow. 
If additional resources are available, include one other school within the same 
community /case study area that have a comprehensive program with a significant 
number of LEP students, but that are not funded with Title VII monies; these might 
be schools that have state /local support and that may be nominated for inclusion in 
the study. 

(b) What are the key issues which threaten the validity of measurement in the study 
(e.g., the level of measurement, appropriateness of measures for LEP students)? 
How can the study best address those issues? 

(c) What are the key issues which will complicate the analyses of the study data (e.g., 
the presence/absence of standards for assessing change, difficulty in drawing 
causal inferences)? How can the study best address those issues? 
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Topic 5: Methodological Issues in Studying School Change 
Written Recommendations from: Diane August 



(a) What are the key issues which complicate the design of the study (e.g., sampling, 
number of measurements)? How can the study best address those issues? 

The key problem is that Congress and the Department will want to know how the 
program is working. That is, if these funds have made a difference in LEP student 
performance. However, given the resources available, do we really want to focus on 
outcomes rather than what we can learn about promising approaches to improving 
practice for LEP students. (There won't be enough money to look at enough schools - 
-random sample of treated schools compared with control schools - in enough depth 
to answer both questions). In addition, it is doubtful that there will be significant 
differences between "treated" and control schools given the within group variance 
that will exist. And if there is, given the resources, all you could probably say is that 
additional money helps. 

A more interesting approach is to study a subset of schools receiving funds and 
examine whether in schools that received these grants, students made progress 
toward meeting higher academic standards. One might also want to look at 
differences in achievement between LEP and nonLEP students within each school, 
broken out by SES status if possible. Absent a control group, one could not attribute 
the improvements to these funds, however. That there was student improvement 
might be sufficient for Congress. In addition, it would be important to document 
what went on these schools so a picture could be "painted" that would help other 
schools (and institutions that provide TA) learn more about what they could do to 
improve services for LEP students (this would help "scale up"). 

(b) What are the key issues which threaten the validity of measurement in the study 
(e.g., the level of measurement, appropriateness of measures for LEP students)? 
How can the study best address those issues? 

One issue is the lack of good measures of L.E.P. student performance in subject areas 
that can be used across sites. One suggested method is to look at how students do, 
given content presented them and also evaluate the quality of the conten' they 
receive. This is difficult to do for more than a limited number of lessons, however, 
and misses what they learn over time. 

Another issue has to do with deciding what to look at, establishing benchmarks for 
success, and figuring out how to measure this. 



23o 



What are the key issues which will complicate the analyses of the study data (e.g., 
the presence/absence of standards for assessing change, difficulty in drawing 
causal inferences)? How can the study best address those issues? 

subsumed in (a) and (b). 
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Topic 5: Methodological Issues in Studying School Change 
Written Recommendations from: Julia Smith 

(a) What are the key issues which complicate the design of the study (e.g., sampling, 
number of measurements)? How can the study best address those issues? 

It is vital to have variety across the dimension you wish to address. If you are 
concerned with schools, you need to have a number of different schools - doing 
things in different ways. If all your schools have a point in common (i.e. all get Title 
VII money), then you cannot form any conclusions about that point of commonality. 

(b) What are the key issues which threaten the validity of measurement in the study 
(e.g., the level of measurement, appropriateness of measures for LEP students)? 
How can the study best address those issues? 

Threats - language vs content of items in measures need to be disentangled. This can 
be dealt with through item design which replicates similar content in different 
languages. 

Threat - Level of measure needs to reflect change rather than constrain it. This can 
be dealt with through linked testing on items which cover a full range of skills, 

Threat - Self-report of attitudes and beliefs is horribly unstable to the point of being 
meaningless. This can be helped by anchoring such reports in behavior and activities, 
i.e. - do you like school -> things you like to do in school (specific). 

(c) What are the key issues which will complicate the analyses of the study data (e.g., 
tlit presence/absence of standards for assessing change, difficulty in drawing 
causal inferences)? How can the study best address those issues? 
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Topic 5: Methodological Issues in Studying School Change 
Written Recommendations from: Judith Torres 

(a) What are the key issues which complicate the design of the study (e.g., sampling, 
number of measurements)? How can the study best address those issues? 

One major issue is that of possible conflicts between the mandates or missions of the 
study - an evaluation of title VII and the desire to contribute to a body of knowledge 
about school change. Sampling plans and study designs may not accommodate both 
equally well. What should the trade-offs be? (A compromise might be to see how 
use of Title VII funds were in fact use to leverage change in various contexts.) This 
will have to be worked out explicitly when the study questions are developed. 

(b) What are the key issues which threaten the validity of measurement in the study 
(e.g., the level of measurement, appropriateness of measures for LEP students)? 
How can the study best address those issues? 

There are many issues here. One will be decisions on defining and operationalizing 
the outcome measures; the issues of valid LEP assessment are included here. Our 
discussion of evaluations of student work in terms of standards will help address this 
issue. Sufficient resources will be key in supporting the development of appropriate 
indicators and evaluation rubrics for judging student work as well as describing 
effective instruction. 

Another challenge will be developing descriptors for the change process, trading off 
between the need to structure the data collection measures while allowing for 
different configurations and understandings to arise during the research process. 

(c) What are the key issues which will complicate the analyses of the study data (e.g., 
the presence/absence of standards for assessing change, difficulty in drawing 
causal inferences)? How can the study best address those issues? 

A major issue is the complexity of contexts and needs. If the purpose of the study 
emerges as understandings of the change process in different contexts, then the 
researchers may find that what emerges is a stream of different case studies and 
patterns for change. Drawing causal inferences is not particularly problematic 
because it is unlikely that anyone will propose to draw them. What the study can 
hopefully do is determine the conditions and processes whereby students have more 
challenging and culturally appropriate instructional experiences because their 
teachers and schools have gone through a process of organizational and personal 
development. 
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Topic 5: Methodological Issues in Studying School Change 
Written Recommendations from: Walter Secada 



(a) What are the key issues which complicate the design of the study (e.g., sampling, 
number of measurements)? How can the study best address those issues? 

(b) What are the key issues which threaten the validity of measurement in the study 
(e.g., the level of measurement, appropriateness of measures for LEP students)? 
How can the study best address those issues? 

(c) What are the key issues which will complicate the analyses of the study data (e.g., 
the presence/absence of standards for assessing change, difficulty in drawing 
causal inferences)? How can the study best address those issues? 

1. Methodologically, the best you can do is to show covariation and document 
processes. This study cannot show "causes" so don't try. 

2. Tracking out what $ pay for - materials, personnel release time, tec. - shows 
the fingerprints for how Title 7 supports comprehensive school services. 

3. Powerful metaphors are helpful for communicating things to various publics - 
e.g. ambitious content; heroic isolate (talking about individual teachers) ; 
collective action 

4. Issues of methodology are so linked to the contexts and questions that it is 
often impossible to list threats. What is a threat in one context is NOT a threat 
in another. 

5. While this is a study of changes, link that change to ambitious pedagogy. 

6. If you're going to score student work, you'll need to develop scales for 
content and scales for language. 



